Augmented Reality for teaching collaborative robots based on a physical simulation

  • Patrick Rückert
  • Finn Meiners
  • Kirsten Tracht
Conference paper


Augmented reality as a technology for programming a collaborative robot is in comparison to existing practice a more intuitive, safe and quick approach. This paper shows a novel concept for connecting a physical robot with its virtual twin, using augmented reality as a visual output and a device for interacting with the simulation model. The core of the system is a station control that contains all interfaces, sequencing logics and databases. For programming the robot, the coordinates are created by the operator by interaction and voice input in the augmented reality environment. All information is stored in a shared database which contains assembly plans and instructions for both workers and robots. A demonstration platform is presented, that allows to implement the concept of the system design for an experimental assembly station for human-robot collaboration. The work station includes a Universal Robot 5® that assists the worker in handling of objects and is operated by coordinates from the station control. A Microsoft Hololens® is used for programming the robot in an augmented reality environment which is based on the output of the physical robot and the work system simulation.


Augmented Reality Simulation Robotics 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andrea Cherubini, A.; Passama, R.; Crosnier, A.; Lasnier, A.; Fraisse, P.: Collaborative manufacturing with physical human–robot interaction. In: Robotics and Computer-Integrated Manufacturing, Volume 40, pp. 1-13, (2016).Google Scholar
  2. 2.
    Michalos, G.; Makris, S.; Papakostas, N.; Mourtzis, D.; Chryssolouris, G.: Automotive assembly technologies review: challenges and outlook for a flexible and adaptive approach. In: CIRP Journal of Manufacturing Science and Technology, 2 (2), pp. 81-91, (2010).Google Scholar
  3. 3.
    Novak-Marcincin, J.; Barna, J.; Janak, M.; Novakova-Marcincinova, L.: Augmented Reality Aided Manufacturing. In: Procedia Computer Science, Volume 25, pp. 23-31, (2013).Google Scholar
  4. 4.
    Eschen, H.; Kötter, T.; Rodeck, R.; Harnisch, M.; Schüppstuhl, T.: Augmented and Virtual Reality for Inspection and Maintenance Processes in the Aviation Industry. In: 6th International Conference on Through-life Engineering Services, TESConf, (2017).Google Scholar
  5. 5.
    Biggs, G., MacDonald, B.: A Survey of Robot Programming Systems. In: Australasian Conference on Robotics and Automation, pp. 1-10, (2003).Google Scholar
  6. 6.
    Magaña, A.; Reinhart, G..: Herstellerneutrale Programmierung von Robotern - Konzept einer Softwarearchitektur zur Steuerung und Programmierung heterogener Robotersysteme. In: wt Werkstattstechnik online, 9, pp. 594-599, (2017).Google Scholar
  7. 7.
    Brecher, C.; Göbel, M.; Herfs, W.; Pohlmann, G.: Roboterprogrammierung durch Demonstration - Ein ganzheitliches Verfahren zur intuitiven Erzeugung und Optimierung von Roboterprogrammen. In: wt Werkstattstechnik online. 9, pp. 655-660, (2009).Google Scholar
  8. 8.
    Chen, S., Ma, H., Yang, C., & Fu, M.Hand gesture based robot station controller using leap motion. In: International Conference on Intelligent Robotics and Applications, Springer International Publishing, pp. 581-591, (2015).Google Scholar
  9. 9.
    Makris, S.; Tsarouchi, P.; Matthaiakis, A. S.; Athanasatos, A.; Chatzigeorgiou, X.; Stefos, M.; Aivaliotis, S.: Dual arm robot in cooperation with humans for flexible assembly. CIRP Annals-Manufacturing Technology, 66, pp.13-16, (2017).Google Scholar
  10. 10.
    P. Tsarouchi, S. Makris, G. Michalos, A.S. Matthaiakis, X. Chatzigeorgiou, A. Athanasatos, M. Stefos, P. Aivaliotis, G. Chryssolouris, G.: ROS Based Coordination of Human Robot Cooperative Assembly Tasks—An Industrial Case Study. In: Procedia CIRP, 37, pp. 254- 259, (2015).Google Scholar
  11. 11.
    Chong, J.W.S., Ong, S.K., Nee, A.Y.C., Youcef-Youmi, K.: Robot programming using augmented reality: an interactive method for planning collision-free paths. In: Robot. Comput. Integr. Manufactoring 25(3), pp. 689–701, (2009).Google Scholar
  12. 12.
    Neea, A.Y.C., Onga, S.K., Chryssolourisb, G., Mourtzisb, D.: Augmented reality applications in design and manufacturing. In: CIRP Annals - Manufacturing Technology 61, pp. 657-679, (2012).Google Scholar
  13. 13.
    Zäh, M. F.; Vogl, W.: Interactive Laser-projection for Programming Industrial Robots. In: Proceedings of the International Symposium on Mixed and Augmented Reality, Santa Barbara, CA: 22–25 October 2006, pp. 125-128, (2006).Google Scholar
  14. 14.
    Andersson, N.; Argyrou, A.; Nägele, F.; Ubis, F.; Esnaola Campos, U.; Ortiz de Zarate, M.; Wilterdink, R.: AR-Enhanced Human-Robot-Interaction - Methodologies, Algorithms, Tools. In: Procedia CIRP, Volume 44, pp. 193-198, (2015).Google Scholar
  15. 15.
    Wünsch, G.: Methoden für die virtuelle Inbetriebnahme automatisierter Produktionssysteme. Zugl.: München, Techn. Univ., Diss., 2007. München: Utz Forschungsberichte/ IWB, 215, (2008).Google Scholar
  16. 16.
    Pentenrieder, K.; Bade, C.; Doil, F.; Meier, P.: Augmented reality-based factory planning - an application tailored to industrial needs. In: ISMAR’07: Proc. 6th Int’l Symp. on Mixed and Augmented Reality, pp. 1–9, (2007).Google Scholar
  17. 17.
    Rückert, P.; Adam, J.; Papenberg, B.; Paulus, H.; Tracht, K.: Assistenzsysteme zur kollaborativen Montage - Mensch-Roboter-Kollaboration basierend auf Bildverarbeitung und Sprachsteuerung. In: wt Werkstattstechnik online, 9, pp. 556-571, (2017).Google Scholar

Copyright information

© Springer-Verlag GmbH Deutschland, ein Teil von Springer Nature 2018

Authors and Affiliations

  • Patrick Rückert
    • 1
  • Finn Meiners
    • 1
  • Kirsten Tracht
    • 2
  1. 1.Bremen Institute for Mechanical EngineeringUniversity of BremenBremenDeutschland
  2. 2.Bremer Institut für Strukturmechanik und ProduktionsanlagenUniversität in Bremen (bime)BremenDeutschland

Personalised recommendations