Towards Generating Simulated Walking Motion Using Position Based Deep Reinforcement Learning
Much of robotics research aims to develop control solutions that exploit the machine’s dynamics in order to achieve an extraordinarily agile behaviour . This, however, is limited by the use of traditional model-based control techniques such as model predictive control and quadratic programming. These solutions are often based on simplified mechanical models which result in mechanically constrained and inefficient behaviour, thereby limiting the agility of the robotic system in development . Treating the control of robotic systems as a reinforcement learning (RL) problem enables the use of model-free algorithms that attempt to learn a policy which maximizes the expected future (discounted) reward without inferring the effects of an executed action on the environment.
KeywordsANYmal Reinforcement learning Walking robot Proximal Policy Optimization
This research is supported by the UKRI and EPSRC (EP/R026084/1, EP/R026173/1, EP/S002383/1) and the EU H2020 project MEMMO (780684). This work has been conducted as part of ANYmal Research, a community to advance legged robotics.
- 1.Gangapurwala, S., et al.: Generative adversarial imitation learning for quadrupedal locomotion using unstructured expert demonstrations (2018)Google Scholar
- 2.Mastalli, C., et al.: Trajectory and foothold optimization using low-dimensional models for rough terrain locomotion (2017)Google Scholar
- 4.Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017)Google Scholar
- 5.Schulman, J., Levine, S., Abbeel, P., Jordan, M., Moritz, P.: Trust region policy optimization. In: International Conference on Machine Learning, pp. 1889–1897, 1 June 2015Google Scholar
- 6.Rohmer, E., Signgh, S. P. N., Freese, M.: V-REP: a versatile and scalable robot simulation framework. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2013)Google Scholar
- 7.Hutter, M., et al.: ANYmal - a highly mobile and dynamic quadrupedal robot. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, pp. 38–44 (2016). https://doi.org/10.1109/IROS.2016.7758092
- 8.Liang, J., et al.: GPU-accelerated robotic simulation for distributed reinforcement learning. CoRL (2018)Google Scholar