Rapid Bipedal Robot Adaptation via Discriminative Internal Model

EasyChair Preprint 10757

Abstract

Reinforcement learning (RL) methods play a crucial role in training bipedal robot locomotion. However, there exists a practical challenge in that well-trained robot policies cannot be directly deployed to different robot dynamics, due to the dynamics gap between the training and the application environment, making the policies inflexible for application in various robot tasks. To address this issue, we propose a rapid adaption framework, named the Discriminative Internal Model (DIM), which
attempts to accelerate the adaption efficiency of RL agents and improve the generalization ability in various dynamic environments. Specifically, DIM first learns a parameterized dynamics
model, called the internal model (IM), in the training environment. In the adaptation phase, the learned IM uses a small number of transitions to quickly adapt to the new environment. The “fine-tuned” IM can simulate rollouts close to the new environment's distribution to speed up policy adaptation. To avoid generating unreliable rollouts that degrade the performance of the policy, we further proposed a state discriminator. It evaluates the reliability of the internal model in each state to determine the number of augmentation rollouts at that state. To demonstrate the effectiveness of the DIM framework, we conduct experiments on a bipedal robot for dynamics transfer and sim-to-real transfer tasks. Extensive experimental evaluations on bipedal locomotion demonstrate that the proposed DIM outperforms the state-of-the-art model-free RL methods.

Keyphrases: Reinforcement Learning, bipedal robot, motion control

@booklet{EasyChair:10757,
  author    = {Zhibo Zhou},
  title     = {Rapid Bipedal Robot Adaptation via Discriminative Internal Model},
  howpublished = {EasyChair Preprint 10757},
  year      = {EasyChair, 2023}}