Skip to main content
SpringerLink
Log in

Human-Augmented Robotic Intelligence (HARI) for Human-Robot Interaction

  • Conference paper
  • First Online:
Proceedings of the Future Technologies Conference (FTC) 2019 (FTC 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1070))

Included in the following conference series:

Abstract

This paper provides a system design framework for a human-robot interaction system. The design introduces a human-augmented robotic intelligence embedded in a human-robot interaction system. The motivations behind the system design are spoken dialogue systems, Wizard-of-OZ framework, and existing HRI designs for socially intelligent robots. In this work, we explain how artificial intelligence of human-robot interaction system is enhanced by human intelligence through collaboration. The collaborative artificial intelligence enables the system to learn from demonstration. The main objective and the gradual progression from this paper is to build an iterative interactive system that is capable of achieving human-robot interaction similar to the nuances of human-human interaction.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 299.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 379.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Dupont, S., Luettin, J.: Audio-visual speech modeling for continuous speech recognition. IEEE Trans. Multimed. 2(3), 141–151 (2000)

    Article  Google Scholar 

  2. Young, S.: Using POMDPS for dialog management. In: 2006 IEEE Spoken Language Technology Workshop (2006)

    Google Scholar 

  3. Zhao, T., Eskenazi, M.: towards end-to-end learning for dialog state tracking and management using deep reinforcement learning. In: Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue (2016)

    Google Scholar 

  4. Wang, Y., Deng, L., Acero, A.: Spoken language understanding- an introduction to the statistical framework. IEEE Signal Process. Mag. 22(5), 16–31 (2005)

    Article  Google Scholar 

  5. Wang, Y., Deng, L., Acero, A.: Semantic frame based spoken language understanding. In: Spoken Language Understanding: Systems for Extracting Semantic Information from Speech, USA, NY, pp. 35–80. Wiley, New York (2011)

    Google Scholar 

  6. Wang, Y.-Y., Acero, A.: Discriminative models for spoken language understanding. In: Proceedings of ICSLP (2006)

    Google Scholar 

  7. Raymond, C., Riccardi, G.: Generative and discriminative algorithms for spoken language understanding. In: Proceedings of Interspeech (2007)

    Google Scholar 

  8. Zue, V., Glass, J.: Conversational interface: advances and challenges. Proc. IEEE 88(8), 1166–1180 (2000)

    Article  Google Scholar 

  9. Mesnil, G., Dauphin, Y., Yao, K., Bengio, Y., Deng, L., Hakkani-Tur, D., He, X., Heck, L., Tur, G., Yu, D., Zweig, G.: Using recurrent neural networks for slot filling in spoken language understanding. IEEE/ACM Trans. Audio Speech Lang. Process. 23(3), 530–539 (2015)

    Article  Google Scholar 

  10. Yaman, S., Deng, L., Yu, D., Wang, Y.-Y., Acero, A.: An integrative and discriminative technique for spoken utterance classification. IEEE Trans. Audio Speech Lang. Process. 16(6), 1207–1214 (2008)

    Article  Google Scholar 

  11. Guo, D., Tur, G., Yih, W., Zweig, G.: Joint semantic utterance classification and slot filling with recursive neural networks. In: 2014 IEEE Spoken Language Technology Workshop (SLT) (2014)

    Google Scholar 

  12. Jaimes, A., Sebe, N.: Multimodal human-computer interaction: a survey. Comput. Vis. Image Underst. 108(1–2), 116–134 (2007)

    Article  Google Scholar 

  13. Chen, L., Huang, T., Miyasato, T., Nakatsu, R.: Multimodal human emotion/expression recognition. In: Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition

    Google Scholar 

  14. Sebe, N., Cohen, I., Gevers, T., Huang, T.: Multimodal approaches for emotion recognition: a survey. In: Internet Imaging VI (2005)

    Google Scholar 

  15. Pietquin, O.: Natural language and dialogue processing. In: Multimodal Signal Processing, pp. 63–92 (2010)

    Google Scholar 

  16. Wen, T., Gasic, M., Mrksic, N., Su, P., Vandyke, D., Young, S.: Semantically conditioned LSTM-based natural language generation for spoken dialogue systems. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (2015)

    Google Scholar 

  17. Fong, T., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots. Robot. Auton. Syst. 42(3–4), 143–166 (2003)

    Article  Google Scholar 

  18. Goodrich, M., Schultz, A.: Human-robot interaction: a survey. In: Foundations and Trends\({\textregistered }\) in Human-Computer Interaction, vol. 1, no. 3, pp. 203–275 (2007)

    Google Scholar 

  19. Moskowitz, G.: Social Cognition. Guildford Press, New York (2005)

    Google Scholar 

  20. Pollack, M.: Intelligent technology for an aging population: the use of AI to assist elders with cognitive impairment. AI Mag. 26(2), 9 (2018)

    Google Scholar 

  21. Pandey, A., Gelin, R.: A mass-produced sociable humanoid robot: pepper: the first machine of its kind. IEEE Robot. Autom. Mag. 25(3), 40–48 (2018)

    Article  Google Scholar 

  22. ABB’s Dual-Arm Collaborative Robot - Industrial Robots From ABB Robotics. New.abb.com (2018). https://new.abb.com/products/robotics/industrial-robots/yumi. Accessed 02 Dec 2018

  23. Amazon Echo: Amazon.com (2018). https://www.amazon.com/dp/B06XCM9LJ4/ref=fs_ods_aucc_rd. Accessed 02 Dec 2018

  24. Google Home: Google Store (2018). https://store.google.com/us/product/google_home?hl=en-US. Accessed 02 Dec 2018

  25. Kim, Y., Provost, E.: Emotion classification via utterance-level dynamics: a pattern-based approach to characterizing affective expressions. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (2013)

    Google Scholar 

  26. Kelley, J.: An iterative design methodology for user-friendly natural language office information applications. ACM Trans. Inf. Syst. 2(1), 26–41 (1984)

    Article  Google Scholar 

  27. Breazeal, C.: Social interactions in HRI: the robot view. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 34(2), 181–186 (2004)

    Article  Google Scholar 

  28. Michalski, R.: Machine learning. Elsevier Science (2014)

    Google Scholar 

  29. Kendall, A., Gal, Y.: What uncertainties do we need in bayesian deep learning for computer vision?. In: Advances in Neural Information Processing Systems 30 (NIPS) (2017)

    Google Scholar 

  30. Barnlund, D.C.: Communication Theory: Second Edition, 2nd edn. Transaction Publishers, New Brunswick (2008)

    Google Scholar 

  31. Kahn, P.H., Freier, N.G., Kanda, T., Ishiguro, H., Ruckert, J.H., Severson, R.L., Kane, S.K.: Design patterns for sociality in human-robot interaction. In: Proceedings of the 3rd International Conference on Human Robot Interaction - HRI 2008 (2008)

    Google Scholar 

  32. Cross, J., Bartley, C., Hamner, E., Nourbakhsh, I.: Arts & bots: application and outcomes of a secondary school robotics program. In: 2015 IEEE Frontiers in Education Conference, 10 (2015)

    Google Scholar 

  33. Mitchell, O.: Can robots save retailers from an apocalypse?, The Robot Report (2018). https://www.therobotreport.com/retail-robots-save-retailers-apocalypse/. Accessed 02 Dec 2018

  34. Baird, N.: Robots, automation and retail: not so cut and dried, Forbes (2018). https://www.forbes.com/sites/nikkibaird/2018/06/19/robots-automation-and-retail-not-so-cut-and-dried/#1ed13fd67b06. Accessed 02 Dec 2018

  35. Underwood, C.: Robots in Retail - Examples of Real Industry Applications | Emerj - Artificial Intelligence Companies, Insights, Research, Emerj (2018). https://emerj.com/ai-sector-overviews/robots-in-retail-examples/. Accessed 02 Dec 2018

  36. CloudMinds – Smart Handheld Raman: Airaman.com (2018). https://www.airaman.com/. Accessed 02 Dec 2018

  37. CloudMinds: En.cloudminds.com (2018). http://en.cloudminds.com/DATA. Accessed 02 Dec 2018

  38. Tian, N., Kuo, B., Ren, X., Yu, M., Zhang, R., Huang, B., Goldberg, K., Sojoudi, S.: A cloud-based robust semaphore mirroring system for social robots. Learning, vol. 12, p. 14

    Google Scholar 

  39. Lin, I.C., Liao, T.C.: A survey of blockchain security issues and challenges. IJ Netw. Secur. 19(5), 653–659 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Aisera, Palo Alto, USA

    Vishwas Mruthyunjaya

  2. Cloudminds Technology, Santa Clara, USA

    Charles Jankowski

Authors
  1. Vishwas Mruthyunjaya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Charles Jankowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Charles Jankowski .

Editor information

Editors and Affiliations

  1. Faculty of Science and Engineering, Saga University, Saga, Japan

    Kohei Arai

  2. The Science and Information (SAI) Organization, Bradford, West Yorkshire, UK

    Rahul Bhatia

  3. The Science and Information (SAI) Organization, Bradford, West Yorkshire, UK

    Supriya Kapoor

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mruthyunjaya, V., Jankowski, C. (2020). Human-Augmented Robotic Intelligence (HARI) for Human-Robot Interaction. In: Arai, K., Bhatia, R., Kapoor, S. (eds) Proceedings of the Future Technologies Conference (FTC) 2019. FTC 2019. Advances in Intelligent Systems and Computing, vol 1070. Springer, Cham. https://doi.org/10.1007/978-3-030-32523-7_14

Download citation

Publish with us

Policies and ethics

Search

Navigation