Abstract
One of the key challenges in robotic bipedal locomotion is finding gait parameters that optimize a desired performance criterion, such as speed, robustness or energy efficiency. Typically, gait optimization requires extensive robot experiments and specific expert knowledge. We propose to apply data-driven machine learning to automate and speed up the process of gait optimization. In particular, we use Bayesian optimization to efficiently find gait parameters that optimize the desired performance metric. As a proof of concept we demonstrate that Bayesian optimization is near-optimal in a classical stochastic optimal control framework. Moreover, we validate our approach to Bayesian gait optimization on a low-cost and fragile real bipedal walker and show that good walking gaits can be efficiently found by Bayesian optimization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The correct notation would be \(\alpha (\hat{f}(\varvec{\theta }))\), but we use \(\alpha (\varvec{\theta })\) for notational convenience.
- 2.
Videos are available at http://www.ias.tu-darmstadt.de/Research/Fox.
References
Auer, P.: Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res. (JMLR) 3, 397–422 (2003)
Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. (JMLR) 13, 281–305 (2012)
Bertsekas, D.P.: Dynamic Programming and Optimal Control, 3rd edn. Athena Scientific, Belmont (2007)
Brochu, E., Cora, V.M., De Freitas, N.:. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599 (2010)
Byrd, R.H., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16(5), 1190–1208 (1995)
Calandra, R., Seyfarth, A., Peters, J., Deisenroth, M.P.: An experimental comparison of Bayesian optimization for bipedal locomotion. In: Proceedings of 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 1951–1958 (2014)
Chernova, S., Veloso, M.: An evolutionary approach to gait learning for four-legged robots. In: Proceedings of Intelligent Robots and Systems (IROS), vol. 3, pp. 2562–2567. IEEE (2004)
Cox, D.D., John, S.: SDO: a statistical method for global optimization. In: Alexandrov, N., Hussaini, M.Y. (eds.) Multidisciplinary Design Optimization: State of the Art, pp. 315–329. SIAM, Philadelpha (1997)
Garnett, R., Osborne, M.A., Roberts, S.J.: Bayesian optimization for sensor set selection. In: Information Processing in Sensor Networks, pp. 209–219. ACM (2010)
Gopalan, N., Deisenroth, M.P., Peters, J.: Feedback error learning for rhythmic motor primitives. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2013)
Grizzle, J.W., Abba, G., Plestan, F.: Asymptotically stable walking for biped robots: analysis via systems with impulse effects. IEEE Trans. Autom. Control 46(1), 51–64 (2001)
Hennig, P., Schuler, C.J.: Entropy search for information-efficient global optimization. J. Mach. Learn. Res. 13, 1809–1837 (2012)
Ijspeert, A.J., Nakanishi, J., Schaal, S.: Learning attractor landscapes for learning motor primitives. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems (NIPS), vol. 15. MIT Press, Cambridge (2003)
Jones, D.R.: A taxonomy of global optimization methods based on response surfaces. J. Global Optim. 21(4), 345–383 (2001)
Jones, D.R., Perttune, C.D., Stuckman, B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theor. Appl. 79(1), 157–181 (1993)
Kober, J., Peters, J.: Learning motor primitives for robotics. In: International Conference on Robotics and Automation (ICRA) (2009)
Kushner, H.J.: A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. J. Basic Eng. 86, 97 (1964)
Lizotte, D., Wang, T., Bowling, M., Schuurmans, D.: Automatic gait optimization with Gaussian process regression. In: Proceedings of International Joint Conferences on Artificial Intelligence (IJCAI), pp. 944–949 (2007)
Mockus, J., Tiesis, V., Zilinskas, A.: The application of Bayesian methods for seeking the extremum. Towards Global Optim. 2, 117–129 (1978)
Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., Kawato, M.: Learning from demonstration and adaptation of biped locomotion. Robot. Autonom. Syst. 47(2), 79–91 (2004)
Osborne, M.A., Garnett, R., Roberts, S.J.: Gaussian processes for global optimization. In: 3rd International Conference on Learning and Intelligent Optimization (LION3), pp. 1–15 (2009)
Pongas, D., Billard, A., Schaal, S.:. Rapid synchronization and accurate phase-locking of rhythmic motor primitives. In: Proceedings of Intelligent Robots and Systems (IROS), pp. 2911–2916. IEEE (2005)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. The MIT Press, Cambridge (2006)
Renjewski, D., Seyfarth, A.: Robots in human biomechanics - a study on ankle push-off in walking. Bioinspir. Biomimetics 7(3), 036005 (2012)
Renjewski, D.: An engineering contribution to human gait biomechanics. Ph.D. Thesis, TU Ilmenau (2012)
Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. In: Advances in Neural Information Processing Systems (NIPS) (2012)
Srinivas, N., Krause, A., Kakade, S., Seeger, M.: Gaussian process optimization in the bandit setting: no regret and experimental design. In: Proceedings of International Conference on Machine Learning (ICML), pp. 1015–1022 (2010)
Tedrake, R., Weirui Zhang, T., Sebastian Seung, H.: Learning to walk in 20 minutes. In: Proceedings of the 14th Yale Workshop on Adaptive and Learning Systems (2005)
Tesch, M., Schneider, J., Choset, H.: Using response surfaces and expected improvement to optimize snake robot gait parameters. In: Proceedings of Intelligent Robots and Systems (IROS), pp. 1069–1074. IEEE (2011)
Acknowledgements
R.C. thanks his father, Enrico Calandra, and Giuseppe Lo Cicero for the invaluable lessons they provided in, among others, life, mechanics and electronics. “Always double-check; then check again.”
The research leading to these results has received funding from the European Community’s Seventh Framework Programme (FP7/2007–2013) under grant agreements #270327 (CompLACS) and #600716 (CoDyCo) and the Department of Computing, Imperial College London.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Calandra, R., Gopalan, N., Seyfarth, A., Peters, J., Deisenroth, M.P. (2014). Bayesian Gait Optimization for Bipedal Locomotion. In: Pardalos, P., Resende, M., Vogiatzis, C., Walteros, J. (eds) Learning and Intelligent Optimization. LION 2014. Lecture Notes in Computer Science(), vol 8426. Springer, Cham. https://doi.org/10.1007/978-3-319-09584-4_25
Download citation
DOI: https://doi.org/10.1007/978-3-319-09584-4_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-09583-7
Online ISBN: 978-3-319-09584-4
eBook Packages: Computer ScienceComputer Science (R0)