Distributed Adaptive Locomotion Learning in ModRED Modular Self-reconfigurable Robot

  • Ayan Dutta
  • Prithviraj Dasgupta
  • Carl Nelson
Part of the Springer Proceedings in Advanced Robotics book series (SPAR, volume 6)


We study the problem of adaptive locomotion learning for modular self-reconfigurable robots (MSRs). MSRs are mostly used in unknown and difficult-to-navigate environments where they can take a completely new shape to accomplish the current task at hand. Therefore it is almost impossible to develop the control sequences for all possible configurations with varying shape and size. The modules have to learn and adapt their locomotion in dynamic time to be more robust in nature. In this paper, we propose a Q-learning based locomotion adaptation strategy which balances exploration versus exploitation in a more sophisticated fashion. We have applied our proposed strategy mainly on the ModRED modular robot within the Webots simulator environment. To show the generalizability of our approach, we have also applied it on a Yamor modular robot. Experimental results show that our proposed technique outperforms a random locomotion strategy and it is able to adapt to module failures.


  1. 1.
    Ahmadzadeh, H., Masehian, E.: Modular robotic systems: methods and algorithms for abstraction, planning, control, and synchronization. Artif. Intell. 223, 27–64 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Baca, J., Hossain, S., Dasgupta, P., Nelson, C.A., Dutta, A.: Modred: hardware design and reconfiguration planning for a high dexterity modular self-reconfigurable robot for extra-terrestrial exploration. Robot. Auton. Syst. 62(7), 1002–1015 (2014)CrossRefGoogle Scholar
  3. 3.
    Baca, J., Woosley, B., Dasgupta, P., Dutta, A., Nelson, C.: Coordination of modular robots by means of topology discovery and leader election: improvement of the locomotion case. In: Distributed Autonomous Robotic Systems, pp. 447–458. Springer, Berlin (2016)Google Scholar
  4. 4.
    Castano, A., Behar, A., Will, P.M.: The conro modules for reconfigurable robots. IEEE/ASME Trans. Mech. 7(4), 403–409 (2002)CrossRefGoogle Scholar
  5. 5.
    Christensen, D.J., Schultz, U.P., Stoy, K.: A distributed strategy for gait adaptation in modular robots. In: 2010 IEEE International Conference on Robotics and Automation (ICRA), pp. 2765–2770. IEEE (2010)Google Scholar
  6. 6.
    Chu, K.D., Hossain, S., Nelson, C.A.: Design of a four-dof modular self-reconfigurable robot with novel gaits. In: ASME 2011 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, pp. 747–754. American Society of Mechanical Engineers (2011)Google Scholar
  7. 7.
    Cully, A., Clune, J., Tarapore, D., Mouret, J.B.: Robots that can adapt like animals. Nature 521(7553), 503–507 (2015)CrossRefGoogle Scholar
  8. 8.
    Dietsch, J., Moeckel, R., Jaquier, C., Drapel, K., Dittrich, E., Upegui, A., Jan Ijspeert, A.: Exploring adaptive locomotion with yamor, a novel autonomous modular robot with bluetooth interface. Ind. Robot. Int. J. 33(4), 285–290 (2006)CrossRefGoogle Scholar
  9. 9.
    Dutta, A., Dasgupta, P.: Simultaneous configuration formation and information collection by modular robotic systems. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 5216–5221 (2016)Google Scholar
  10. 10.
    Ijspeert, A.J.: Central pattern generators for locomotion control in animals and robots: a review. Neural Netw. 21(4), 642–653 (2008)CrossRefGoogle Scholar
  11. 11.
    Kamimura, A., Kurokawa, H., Yoshida, E., Murata, S., Tomita, K., Kokaji, S.: Automatic locomotion design and experiments for a modular robotic system. IEEE/ASME Trans. Mech. 10(3), 314–325 (2005)CrossRefGoogle Scholar
  12. 12.
    Kamimura, A., Kurokawa, H., Yoshida, E., Tomita, K., Kokaji, S., Murata, S.: Distributed adaptive locomotion by a modular robotic system, m-tran ii. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2004. (IROS 2004). Proceedings, vol. 3, pp. 2370–2377. IEEE (2004)Google Scholar
  13. 13.
    Kapetanakis, S., Kudenko, D.: Reinforcement learning of coordination in cooperative multi-agent systems. AAAI/IAAI 2002, 326–331 (2002)zbMATHGoogle Scholar
  14. 14.
    Nutt, J., Marsden, C., Thompson, P.: Human walking and higher-level gait disorders, particularly in the elderly. Neurology 43(2), 268–268 (1993)CrossRefGoogle Scholar
  15. 15.
    Shen, W.M., Salemi, B., Will, P.: Hormone-inspired adaptive communication and distributed control for conro self-reconfigurable robots. IEEE Trans. Robot. Autom. 18(5), 700–712 (2002)CrossRefGoogle Scholar
  16. 16.
    Shen, W.M., Will, P.: Docking in self-reconfigurable robots. In: 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2001. Proceedings, vol. 2, pp. 1049–1054. IEEE (2001)Google Scholar
  17. 17.
    Sproewitz, A., Moeckel, R., Maye, J., Ijspeert, A.J.: Learning to move in modular robots using central pattern generators and online optimization. Int. J. Robot. Res 27(3–4), 423–443 (2008)CrossRefGoogle Scholar
  18. 18.
    Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction. MIT Press, Cambridge (1998)Google Scholar
  19. 19.
    Yim, M.: Locomotion with a unit-modular reconfigurable robot. Ph.D. thesis, Citeseer (1994)Google Scholar
  20. 20.
    Yim, M., Shen, W.M., Salemi, B., Rus, D., Moll, M., Lipson, H., Klavins, E., Chirikjian, G.S.: Modular self-reconfigurable robot systems [grand challenges of robotics]. IEEE Robot. Autom. Mag. 14(1), 43–52 (2007).

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Computer Science DepartmentUniversity of Nebraska OmahaOmahaUSA
  2. 2.Mechanical and Materials Engineering DepartmentUniversity of Nebraska LincolnLincolnUSA

Personalised recommendations