Robot Learning for Persistent Autonomy
Autonomous robots are not very good at being autonomous. They work well in structured environments, but fail quickly in the real world facing uncertainty and dynamically changing conditions. In this chapter, we describe robot learning approaches that help to elevate robot autonomy to the next level, the so-called ‘persistent autonomy’. For a robot to be ‘persistently autonomous’ means to be able to perform missions over extended time periods (e.g. days or months) in dynamic, uncertain environments without need for human assistance. In particular, persistent autonomy is extremely important for robots in difficult-to-reach environments such as underwater, rescue, and space robotics. There are many facets of persistent autonomy, such as: coping with uncertainty, reacting to changing conditions, disturbance rejection, fault tolerance, energy efficiency and so on. This chapter presents a collection of robot learning approaches that address many of these facets. Experiments with robot manipulators and autonomous underwater vehicles demonstrate the usefulness of these learning approaches in real world scenarios.
KeywordsFuzzy System Autonomous Underwater Vehicle Autonomous Robot Iterative Learn Control Imitation Learning
We would like to thank Professor David Lane from the Ocean Systems Laboratory, Heriot-Watt University, UK, for introducing us to the topic of persistent autonomy.
We are grateful to Arnau Carrera, Narcís Palomeras, and Marc Carreras from the Computer Vision and Robotics Group (VICOROB), University of Girona, Spain, for making it possible to conduct real-world experiments with the Girona 500 AUV.
This work was supported by the European project PANDORA: Persistent Autonomy through learNing, aDaptation, Observation and ReplAnning, contract FP7-ICT-288273 (PANDORA 2012).
- Ahmadzadeh SR, Kormushev P, Caldwell DG (2013a) Autonomous robotic valve turning: a hierarchical learning approach. In: 2013 IEEE international conference on robotics and automation (ICRA). IEEE, pp 4614–4619Google Scholar
- Ahmadzadeh SR, Leonetti M, Kormushev P (2013b) Online direct policy search for thruster failure recovery in autonomous underwater vehicles. In: 6th international workshop on evolutionary and reinforcement learning for autonomous robot system (ERLARS 2013), Taormina, ItalyGoogle Scholar
- Ahmadzadeh SR, Jamisola RS, Kormushev P, Caldwell DG (2014a) Learning reactive robot behavior for autonomous valve turning. In: Proceedings of the IEEE international conference on humanoid robots (Humanoids 2014), Madrid, SpainGoogle Scholar
- Ahmadzadeh SR, Leonetti M, Carrera A, Carreras M, Kormushev P, Caldwell DG (2014b) Online discovery of AUV control policies to overcome thruster failure. In: 2014 IEEE international conference on robotics and automation (ICRA). IEEE, pp 6522–6528Google Scholar
- Ajoudani A, Lee J, Rocchi A, Ferrati M, Mingo E, Settimi A, Caldwell DG, Bicchi A, Tsagarakis N (2014) A manipulation framework for compliant humanoid COMAN: application to a valve turning task. In: 2014 IEEE-RAS international conference on humanoid robots (Humanoids 2014). IEEE, pp 664–670Google Scholar
- Alessandri A, Caccia M, Veruggio G (1998) A model-based approach to fault diagnosis in unmanned underwater vehicles. In: OCEANS’98 conference proceedings, vol 2. IEEE, pp 825–829Google Scholar
- Alunni N, Phillips-Grafftin C, Suay HB, Lofaro D, Berenson D, Chernova S, Lindeman RW, Oh P (2013) Toward a user-guided manipulation framework for high-dof robots with limited communication. In: 2013 IEEE international conference on technologies for practical robot applications (TePRA). IEEE, pp 1–6Google Scholar
- Anisi DA, Persson E, Heyer C (2011) Real-world demonstration of sensor-based robotic automation in oil & gas facilities. In: 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 235–240Google Scholar
- Anisi DA, Skourup C, Petrochemicals A (2012) A step-wise approach to oil and gas robotics. In: IFAC workshop on automatic control in offshore oil and gas production, Trondheim, Norway, vol 31Google Scholar
- Antonelli G (2006) Underwater robots: motion and force control of vehicle-manipulator systems. Springer tracts in advanced robotics. Springer, New YorkGoogle Scholar
- Caccia M, Bono R, Bruzzone G, Bruzzone G, Spirandelli E, Veruggio G (2001) Experiences on actuator fault detection, diagnosis and accomodation for ROVs. In: International symposiyum of unmanned untethered sub-mersible technologyGoogle Scholar
- Carrera A, Ahmadzadeh S, Ajoudani A, Kormushev P, Carreras M, Caldwell D (2012) Towards autonomous robotic valve turning. Cybern Inf Technol 12(3):17–26Google Scholar
- Cheng ASF, Leonard NE (1999) Fin failure compensation for an unmanned underwater vehicle. In: Proceedings of the 11th international symposium on unmanned untethered submersible technologyGoogle Scholar
- Hamilton K, Lane D, Taylor N, Brown K (2001) Fault diagnosis on autonomous robotic vehicles with recovery: an integrated heterogeneous-knowledge approach. In: Proceedings of the IEEE International Conference on Robotics and Automation, ICRA, 2001, vol 4. IEEE, pp 3232–3237Google Scholar
- Karras GC, Bechlioulis CP, Leonetti M, Palomeras N, Kormushev P, Kyriakopoulos KJ, Caldwell DG (2013) On-line identification of autonomous underwater vehicles through global derivative-free optimization. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS)Google Scholar
- Konidaris G, Osentoski S, Thomas PS (2011) Value function approximation in reinforcement learning using the fourier basis. In: AAAIGoogle Scholar
- Kormushev P, Caldwell DG (2013a) Improving the energy efficiency of autonomous underwater vehicles by learning to model disturbances. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS), Tokyo, JapanGoogle Scholar
- Kormushev P, Caldwell DG (2013b) Towards improved AUV control through learning of periodic signals. In: Proceedings of the MTS/IEEE international conference on OCEANS 2013, San DiegoGoogle Scholar
- Lane DM, Maurelli F, Kormushev P, Carreras M, Fox M, Kyriakopoulos K (2012) Persistent autonomy: the challenges of the PANDORA project. In: Proceedings of the IFAC MCMCGoogle Scholar
- Leonetti M, Kormushev P, Sagratella S (2012) Combining local and global direct derivative-free optimization for reinforcement learning. Cybern Inf Technol 12(3):53–65Google Scholar
- Leonetti M, Ahmadzadeh SR, Kormushev P (2013) On-line learning to recover from thruster failures on autonomous underwater vehicles. In: OCEANS 2013. IEEEGoogle Scholar
- Moore KL (2012) Iterative learning control for deterministic systems. Springer Science & Business Media, LondonGoogle Scholar
- Orsag M, Korpela C, Bogdan S, Oh P (2014) Valve turning using a dual-arm aerial manipulator. In: 2014 international conference on unmanned aircraft systems (ICUAS). IEEE, pp 836–841Google Scholar
- PANDORA (2012) Persistent autonomy through learning, adaptation, observation and re-planning. http://persistentautonomy.com/, PANDORA European Project
- Perrault D, Nahon M (1998) Fault-tolerant control of an autonomous underwater vehicle. In: OCEANS’98 conference proceedings, vol 2. IEEE, pp 820–824Google Scholar
- Podder T, Antonelli G, Sarkar N (2000) Fault tolerant control of an autonomous underwater vehicle under thruster redundancy: simulations and experiments. In: Proceedings of the IEEE international conference on robotics and automation, ICRA’00, 2000, vol 2. IEEE, pp 1251–1256Google Scholar
- Wang L (1999) A course on fuzzy systems. Prentice-Hall press, Upper Saddle RiverGoogle Scholar