Robot Online Learning Through Digital Twin Experiments: A Weightlifting Project

  • Igor VernerEmail author
  • Dan Cuperman
  • Amy Fang
  • Michael Reitman
  • Tal Romm
  • Gali Balikin
Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 22)


This paper proposes and explores an approach in which robotics projects of novice engineering students focus on development of learning robots. We implemented a reinforcement learning scenario in which a humanoid robot learns to lift a weight of unknown mass through autonomous trial-and-error search. To expedite the process, trials of the physical robot are substituted by simulations with its virtual twin. The optimal parameters of the robot posture for executing the weightlifting task, found by analysis of the virtual trials, are transmitted to the robot through internet communication. The approach exposes students to the concepts and technologies of machine learning, parametric design, digital prototyping and simulation, connectivity and internet of things. Pilot implementation of the approach indicates its potential for teaching freshman and HS students, and for teacher education.


Robot learning Weightlifting Virtual twin Internet of Things 


  1. 1.
    Peters, J., Lee, D.D., Kober, J., Nguyen-Tuong, D., Bagnell, D., Schaal, S.: Robot learning. In: Siciliano, B., Khatib, O. (eds.) Springer Handbook of Robotics, pp. 357–394. Springer (2017)Google Scholar
  2. 2.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  3. 3.
    Kormushev, P., Calinon, S., Caldwell, D.G.: Reinforcement learning in robotics: applications and real-world challenges. Robotics 2(3), 122–148 (2013)CrossRefGoogle Scholar
  4. 4.
    Nguyen-Tuong, D., Peters, J.: Model learning for robot control: a survey. Cogn. Process. 12(4), 319–340 (2011)CrossRefGoogle Scholar
  5. 5.
    Reckhaus, M., Hochgeschwender, N., Paulus, J., Shakhimardanov, A., Kraetzschmar, G.K.: An overview about simulation and emulation in robotics. In: Proceedings of SIMPAR, pp. 365–374 (2010)Google Scholar
  6. 6.
    Kehoe, B., Patil, S., Abbeel, P., Goldberg, K.: A survey of research on cloud robotics and automation. IEEE Trans. Autom. Sci. Eng. 12(2), 398–409 (2015)CrossRefGoogle Scholar
  7. 7.
    Verner, I., Cuperman, D., Krishnamachar, A., Green S.: Learning with learning robots: a weight-lifting project. In: Robot Intelligence Technology and Applications, vol. 4, pp. 319–327. Springer (2017)Google Scholar
  8. 8.
    Harada, K., Kajita, S., Saito, H., Morisawa, M., Kanehiro, F., Fujiwara, K., Kaneko, K., Hirukawa, H.: A humanoid robot carrying a heavy object. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1712–1717 (2005)Google Scholar
  9. 9.
    Arisumi, H., Miossec, S., Chardonnet, J.R., Yokoi, K.: Dynamic lifting by whole body motion of humanoid robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 668–675 (2008)Google Scholar
  10. 10.
    Rosenstein, M.T., Barto, A.G., Van Emmerik, R.E.: Learning at the level of synergies for a robot Weightlifter. Robot. Auton. Syst. 54(8), 706–717 (2006)CrossRefGoogle Scholar
  11. 11.
    Kuo, C.H., Kuo, Y.C., Chen, T.S.: Process modeling and task execution of FIRA weight-lifting games with a humanoid robot. In: Conference Towards Autonomous Robotic Systems, pp. 354–365. Springer, Heidelberg (2012)Google Scholar
  12. 12.
    Michieletto, S., Tosello, E., Pagello, E., Menegatti, E.: Teaching humanoid robotics by means of human teleoperation through RGB-D sensors. Robot. Auton. Syst. 75, 671–678 (2016)CrossRefGoogle Scholar
  13. 13.
    Anderson, J., Baltes, J., Cheng, C.T.: Robotics competitions as benchmarks for AI research. Knowl. Eng. Rev. 26(1), 11–17 (2011)CrossRefGoogle Scholar
  14. 14.
    Ahlgren, D.J., Verner, I.M.: Socially responsible engineering education through assistive robotics projects: the RoboWaiter competition. Int. J. Soc. Robot. 5(1), 127–138 (2013)CrossRefGoogle Scholar
  15. 15.
    Verner, I.M., Cuperman, D., Cuperman, A., Ahlgren, D., Petkovsek, S., Burca, V.: Humanoids at the assistive robot competition RoboWaiter 2012. In: Robot Intelligence Technology and Applications 2012, pp. 763–774. Springer, Heidelberg (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Igor Verner
    • 1
    Email author
  • Dan Cuperman
    • 1
  • Amy Fang
    • 2
  • Michael Reitman
    • 3
  • Tal Romm
    • 1
    • 3
  • Gali Balikin
    • 1
    • 3
  1. 1.Technion – Israel Institute of TechnologyHaifaIsrael
  2. 2.Massachusetts Institute of TechnologyBostonUSA
  3. 3.PTC Inc.HaifaIsrael

Personalised recommendations