Abstract
Learning robots that can acquire new motor skills and refine existing one has been a long standing vision of robotics, artificial intelligence, and the cognitive sciences. Early steps towards this goal in the 1980s made clear that reasoning and human insights will not suffice. Instead, new hope has been offered by the rise of modern machine learning approaches. However, to date, it becomes increasingly clear that off-the-shelf machine learning approaches will not suffice for motor skill learning as these methods often do not scale into the high-dimensional domains of manipulator and humanoid robotics nor do they fulfill the real-time requirement of our domain. As an alternative, we propose to break the generic skill learning problem into parts that we can understand well from a robotics point of view. After designing appropriate learning approaches for these basic components, these will serve as the ingredients of a general approach to motor skill learning. In this paper, we discuss our recent and current progress in this direction. For doing so, we present our work on learning to control, on learning elementary movements as well as our steps towards learning of complex tasks. We show several evaluations both using real robots as well as physically realistic simulations.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Sciavicco, L., Siciliano, B.: Modeling and control of robot manipulators. MacGraw-Hill, Heidelberg (2007)
Schaal, S., Atkeson, C.G., Vijayakumar, S.: Scalable techniques from nonparameteric statistics for real-time robot learning. Applied Intelligence, 49–60 (2002)
Schaal, S., Ijspeert, A.J., Billard, A.: Computational approaches to motor learning by imitation. In: Frith, C.D., Wolpert, D. (eds.) The Neuroscience of Social Interaction, pp. 199–218. Oxford University Press, Oxford (2004)
Ijspeert, A.J., Nakanishi, J., Schaal, S.: Learning attractor landscapes for learning motor primitives. In: Advances in Neural Information Processing Systems, vol. 15, pp. 1547–1554. MIT Press, Cambridge (2003)
Nakanishi, J., Farrell, J.A., Schaal, S.: Composite adaptive control with locally weighted statistical learning. Neural Networks 18(1), 71–90 (2005)
Nguyen-Tuong, D., Seeger, M., Peters, J.: Local Gaussian Process Regression for Real Time Online Model Learning and Control. In: Advances in Neural Information Processing Systems (NIPS 2008), vol. 21. MIT Press, Cambridge (2009)
Nguyen-Tuong, D., Seeger, M., Peters, J.: Computed torque control with nonparametric regression models. In: Proceedings of the 2008 American Control Conference (ACC 2008) (2008)
Nguyen-Tuong, D., Peters, J.: Semi-parametric regression in learning inverse dynamics. In: International Conference on Robotics & Automation (ICRA) (2010) (Submitted)
Chiappa, S., Peters, J.: Motion segmentation by detecting in continuous time-series. In: Advances in Neural Information Processing Systems (NIPS 2009), vol. 22. MIT Press, Cambridge (2010) (Submitted)
Chiappa, S., Kober, J., Peters, J.: Using Bayesian Dynamical Systems for Motion Template Libraries. In: Advances in Neural Information Processing Systems (NIPS 2008), vol. 21. MIT Press, Cambridge (2009)
Dayan, P., Hinton, G.E.: Using expectation-maximization for reinforcement learning. Neural Computation 9(2), 271–278 (1997)
Peters, J., Schaal, S.: Reinforcement learning of motor skills with policy gradients. Neural Networks 21(4), 682–697 (2008)
Kober, J., Peters, J.: Policy Search for Motor Primitives in Robotics. In: Advances in Neural Information Processing Systems (NIPS 2008), vol. 21. MIT Press, Cambridge (2009)
Kober, J., Mohler, B., Peters, J.: Learning Perceptual Coupling for Motor Primitives. In: Proceedings of the IEEE International Conference on Intelligent RObots and Systems (IROS) (2008)
Kober, J., Peters, J.: Learning Motor Primitives for Robotics. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2009)
Jordan, M., Jacobs, R.: Hierarchical mixture of experts and the EM algorithm. Neural Computation 6, 181–214 (1994)
Muelling, K.: Learning (2009)
Peters, J., Mistry, M., Udwadia, F.E., Nakanishi, J., Schaal, S.: A unifying methodology for robot control with redundant DOFs. Autonomous Robots 24(1), 1–12 (2008)
Peters, J., Schaal, S.: Reinforcement learning by reward-weighted regression for operational space control. In: Proceedings of the International Conference on Machine Learning (ICML 2007), Oregon, USA, pp. 745–750 (2007)
Peters, J., Schaal, S.: Learning to Control in Operational Space. The International Journal of Robotics Research 27(2), 197–212 (2008)
Pastor, P., Hoffmann, H., Asfour, T., Schaal, S.: Learning and generalization of motor skills by learning from demonstration. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2009)
Peters, J., Nguyen, D.: Real-Time Learning of Resolved Velocity Control on a Mitsubishi PA-10. In: International Conference on Robotics and Automation (ICRA) (2008)
Sentis, L., Khatib, O.: Synthesis of whole-body behaviors through hierarchical control of behavioral primitives. International Journal of Humanoid Robotics 2(4), 505–518 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Peters, J., Mülling, K., Kober, J., Nguyen-Tuong, D., Krömer, O. (2011). Towards Motor Skill Learning for Robotics. In: Pradalier, C., Siegwart, R., Hirzinger, G. (eds) Robotics Research. Springer Tracts in Advanced Robotics, vol 70. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19457-3_28
Download citation
DOI: https://doi.org/10.1007/978-3-642-19457-3_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19456-6
Online ISBN: 978-3-642-19457-3
eBook Packages: EngineeringEngineering (R0)