Skip to main content
SpringerLink
Log in
Book cover

Advances in Robot Design and Intelligent Control pp 255–263Cite as

Designing a Multimodal Human-Robot Interaction Interface for an Industrial Robot

  • Conference paper
  • First Online:

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 371))

Abstract

This paper presents a framework for multimodal human-robot interaction. The proposed framework is intended to bring important contributions to the development of human robot interaction to facilitate intuitive programming and to enable easily adapting to changes in robot task without the need of using skilled personnel. The key elements of this system are speech and hand gesture recognition, text programming, and interaction capabilities that allow the user to take over the control of the robot at any given time. Furthermore, our approach is focused on robot tasks. A user can express his/her preference for one or more modalities of interaction so that selected modalities fit user’s personal needs.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Mocan, B., Fulea, M., Brad, S.: Framework for developing a multimodal programming interface used on industrial robots. Robotica Manage. 19(2), 29–33 (2014)

    Google Scholar 

  2. Fulea, M., Popescu, S., Brad, E., Mocan, B., Murar, M.: A literature survey on reconfigurable industrial robotic work cells. In: Proceedings of the 6th International Conference on Robotics ROBOTICS’14. Bucharest (2014)

    Google Scholar 

  3. Mocan, B., Buna, D., Fulea, M., Brad, S.: Increasing the efficiency of robotic manufacturing systems by layout optimization. In: Proceedings of the 6th International Conference on Robotics ROBOTICS’14. Bucharest (2014)

    Google Scholar 

  4. Kongn, J., Zhang, W.Y., Yu, N., Xia, X.J.: Design of human-centric adaptive multimodal interfaces. Int. J. Human-Comput. Stud. 69, 854–869 (2011)

    Google Scholar 

  5. Brad, S., Mocan, B., Brad, E., Fulea, M.: Leading innovation to improve complex process performances by systematic problem analysis with TRIZ, TFC2014, Global Innovation Convention, EPFL, Lausanne, Procedia CIRP. Elsevier, Amsterdam (2014)

    Google Scholar 

  6. Akan, B., Ameri, A., Curuklu, B., Asplund, L.: Intuitive industrial robot programming through incremental multimodal language and augmented reality, 2011. In: IEEE International Conference on Robotics and Automation, pp. 3934–3939. Shanghai (2011)

    Google Scholar 

  7. Mocan, B., Fulea, M., Brad, E., Brad, S.: State-of-the-Art and proposals on reducing energy consumption in the case of industrial robotic systems. In: Proceedings of the 2014 International Conference on Production Research—Regional Conference Africa, Europe and the Middle East, pp. 328–334. Cluj-Napoca, Romania, 1–5 July (2014)

    Google Scholar 

  8. Turk, M.: Multimodal interaction: A review. Pattern Recogn. Lett. 36, 189–195 (2014)

    Article  Google Scholar 

  9. Hirzinger, G., Bals, J., Otter, M., Stelter, J.: The DLR-KUKA success story: Robotics research improves industrial robots. IEEE Robot. Autom. Mag. 12(3), 16–23 (2005)

    Google Scholar 

  10. Mocan, B.: Performance planning of arc welding robotic systems using specific tools for quality planning and systematic introduction of innovation—part I, II, III. J. Calitatea—acces la success 13(127), 80–85 (2012)

    Google Scholar 

  11. Mocan, B., Fulea, M.: Offline Programming of Robotic Arc Welding Systems. Acta Technica Napocensis, S: Appl. Math. Mech. 1(54), 173–178 (2011)

    Google Scholar 

  12. Brick, T., Scheutz, M.: Incremental natural language processing for HRI. In: Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction—HRI ’07, 261–265 (2007)

    Google Scholar 

  13. Brad, S., Fulea, M., Brad, E., Mocan, B.: Smart deployment of demonstrators into successful commercial solutions. In: Proceedings of the 24th CIRP Design Conference, pp. 165–175. Milan, Italy, 14–16 March (2014)

    Google Scholar 

  14. Horst, P., Schreck, G., Willnow, C.: SME-Service networks for cooperative operation of robot installations. In: International Conference Emerging Solutions for Future Manufacturing Systems, pp. 339–346 (2006)

    Google Scholar 

  15. Marin, R., Sanz, P., Nebot, P., Wirz, R.: A multimodal interface to control a robot arm via the web: a case study on remote programming. IEEE Trans. Ind. Electron. 52(6), 1506–1520 (2005)

    Google Scholar 

  16. Dumas, B., Lalanne, D., Oviatt, S.: Multimodal interfaces: A survey of principles, models and frameworks, Human Machine Interaction. Lecture Notes Computer Science, vol. 5440, pp. 3–26. Springer, Berlin (2009)

    Google Scholar 

  17. Van Wassenhove, V., Grant, K.W., Poeppel, D.: Visual speech speeds up the neural processing of auditory speech. In: Proc. Nat. Acad. Sci. 102, 1181–1186 (2005)

    Google Scholar 

  18. Bolt, R.A.: Put-that-there: Voice and gesture at the graphics interface. Int. Conf. Comput. Graph. Interact. Tech. 14, 261–264 (1980)

    Google Scholar 

  19. Xiao, B., Girand, C., Oviatt, S.L.: Multimodal integration patterns in children. In: International Conference on Spoken Language Processing, pp. 629–632 (2002)

    Google Scholar 

  20. Oviatt, S.L., Lunsford, R., Coulston, R.: Individual differences in multimodal integration patterns: What are they and why do they exist? In ACM SIGCHI Conference on Human Factors in Computing Systems, vol. 2, No. 7, 241–249 (2005)

    Google Scholar 

  21. Bohus, D., Horvitz, E.: Facilitating multiparty dialog with gaze, gesture, and speech. In: ACM International Conference on Multimodal Interfaces (ICMIMLMI). Beijing (2010)

    Google Scholar 

  22. Johnston, M., Bangalore, S: Finite-state multimodal parsing and understanding. In: Proceedings of the 18th Conference on Computational Linguistics, pp. 369–375 (2000)

    Google Scholar 

  23. Pires, J.N.: Robotics for small and medium enterprises: control and programming challenges, Industrial Robot (2006)

    Google Scholar 

  24. Fuegen, C., Holzapfel, H., Waibel, A.: Tight coupling of speech recognition and dialog management dialog-context dependent grammar weighting for speech recognition. In: 8th International Conference on Spoken Language Processing, pp. 2–5 (2004)

    Google Scholar 

  25. Schlangen, D., Skantze, G.: A general, abstract model of incremental dialogue processing. In: European Chapter Meeting of the ACL, 710–718 (2009)

    Google Scholar 

  26. Hsiao, K.-Y., Vosoughi, S., Tellex, S., Kubat, R., Roy, D.: Object schemas for responsive robotic language use. In: Proceedings of the 3rd International Conference on Human Robot Interaction—HRI ’08, pp. 231–235 (2008)

    Google Scholar 

  27. Iba, S., Paredis, C.J.J., Khosla, P.K.: Interactive multi-modal robot programming. In: Proceedings 2002 IEEE Intetnational Conference on Robotics and Automation, pp. 161–168 (2002)

    Google Scholar 

  28. McGuire, P.C., Fritsch, J., Steil, J.J., Roethling, F., Fink, G.A., Wachsmuth, S., Sagerer, G., Ritter, H.: Multi-modal human-machine communication for instructing robot grasping tasks. In: International Conference on Intelligent Robots and Systems (IROS), pp. 1082–1089 (2002)

    Google Scholar 

  29. Fang, H., Ong, S.K., Nee, A.Y.-C.: Robot Programming Using Augmented Reality, IEEE, (2009)

    Google Scholar 

  30. Jara, C.A., Candelas, F.A., Gil, P., Fernandez, M., Torres, F.: An augmented reality interface for training robotics through the web. Communication, pp. 189–194 (2005)

    Google Scholar 

  31. Oviatt, S.L.: Ten myths of multimodal interaction. Commun. ACM 42(11), 74–81 (1999)

    Article  Google Scholar 

  32. Marin, R., Sanz, P., Sanchez, J.: A very high level interface to tele-operate a robot via Web including augmented reality. In: Proceedings of IEEE International Conference on Robotics and Automation, (May), pp. 2725–2730 (2002)

    Google Scholar 

  33. Quek, F.: Toward a Vision-Based Hand Gesture Interface, Virtual Reality System Technology Conference, pp. 17–29. Singapore (1994)

    Google Scholar 

Download references

Acknowledgments

Support within the project POSDRU/159/1.5/S/137516 PARTING is acknowledged with gratitude.

Author information

Authors and Affiliations

  1. Department of Engineering Design and Robotics, Technical University of Cluj-Napoca, Cluj-Napoca, Romania

    Bogdan Mocan, Mircea Fulea & Stelian Brad

Authors
  1. Bogdan Mocan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mircea Fulea
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stelian Brad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bogdan Mocan .

Editor information

Editors and Affiliations

  1. Fac of Auto Ctrl & Comp Sci (Automatica), University Politehnica of Bucharest, Bucharest, Romania

    Theodor Borangiu

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Mocan, B., Fulea, M., Brad, S. (2016). Designing a Multimodal Human-Robot Interaction Interface for an Industrial Robot. In: Borangiu, T. (eds) Advances in Robot Design and Intelligent Control. Advances in Intelligent Systems and Computing, vol 371. Springer, Cham. https://doi.org/10.1007/978-3-319-21290-6_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-21290-6_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-21289-0

  • Online ISBN: 978-3-319-21290-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation