Skip to main content
SpringerLink
Log in
Book cover

Pacific Rim Conference on Multimedia

PCM 2015: Advances in Multimedia Information Processing -- PCM 2015 pp 181–190Cite as

A Survey on Media Interaction in Social Robotics

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9315))

Abstract

Social robots have attracted increasing research interests in academic and industry communities. The emerging media technologies greatly inspired human-robot interaction approaches, which aimed to tackle important challenges in practical applications. This paper presents a survey of recent works on media interaction in social robotics. We first introduce the state-of-the-art social robots and the related concepts. Then, we review the visual interaction approaches through various human actions such as facial expression, hand gesture and body motion, which have been widely considered as effective media interaction ways with robots. Furthermore, we summarize the event detection approaches which are crucial for robots to understand the environment and human intentions. While the emphasis is on vision-based interaction approaches, the multimodal interaction works are also briefly summarized for practitioners.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Aigner, P., McCarragher, B.J.: Modeling and constraining human interactions in shared control utilizing a discrete event framework. IEEE Trans. Syst. Man Cybern. Part A: Syst. Hum. 30(3), 369–379 (2000)

    Article  Google Scholar 

  2. Alonso-Martin, F., Malfaz, M., Sequeira, J., Gorostiza, J.F., Salichs, M.A.: A multimodal emotion detection system during human-robot interaction. Sens. 13(11), 15549–15581 (2013)

    Article  Google Scholar 

  3. Alves, S., Silva, I., Ranieri, C., Ferasoli Filho, H.: Assisted robot navigation based on speech recognition and synthesis. In: ISSNIP-IEEE Biosignals and Biorobotics Conference (2014)

    Google Scholar 

  4. Argall, B.D., Billard, A.G.: A survey of tactile human-robot interactions. Rob. Auton. Syst. 58(10), 1159–1176 (2010)

    Article  Google Scholar 

  5. Baxter, P.E., de Greeff, J., Belpaeme, T.: Cognitive architecture for humancrobot interaction: towards behavioural alignment. Biol. Inspired Cogn. Archit. 6, 30–39 (2013)

    Google Scholar 

  6. Van den Bergh, M., Carton, D., de Nijs, R., Mitsou, N., Landsiedel, C., Kuehnlenz, K., Wollherr, D., Van Gool, L., Buss, M.: Real-time 3D hand gesture interaction with a robot for understanding directions from humans. In: IEEE RO-MAN (2011)

    Google Scholar 

  7. Berghofer, E., Schulze, D., Rauch, C., Tscherepanow, M., Kohler, T., Wachsmuth, S.: ART-based fusion of multi-modal perception for robots. Neurocomput. 107, 11–22 (2013)

    Article  Google Scholar 

  8. Breazeal, C.: Designing Sociable Robots. MIT Press, Cambridge (2002)

    MATH  Google Scholar 

  9. Cabibihan, J.J., Javed, H., Ang, M.J., Aljunied, S.M.: Why robots? a survey on the roles and benefits of social robots in the therapy of children with Autism. Int. J. Soc. Rob. 5(4), 593–618 (2013)

    Article  Google Scholar 

  10. Cheng, H., Luo, J., Chen, X.: A windowed dynamic time warping approach for 3D continuous hand gesture recognition. In: IEEE ICME (2014)

    Google Scholar 

  11. Cheng, H., Yu, R., Liu, Z., Yang, L., Chen, X.: Kernelized pyramid nearest-neighbor search for object categorization. Mach. Vis. Appl. 25(4), 931–941 (2014)

    Article  Google Scholar 

  12. Costa, S., Soares, F., Santos, C.: Facial expressions and gestures to convey emotions with a humanoid robot. In: Herrmann, G., Pearson, M.J., Lenz, A., Bremner, P., Spiers, A., Leonards, U. (eds.) ICSR 2013. LNCS, vol. 8239, pp. 542–551. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  13. Coutinho, F.L., Morimoto, C.H.: Improving head movement tolerance of cross-ratio based eye trackers. IJCV 101(3), 459–481 (2013)

    Article  Google Scholar 

  14. Devillers, L., Tahon, M., Sehili, M., Delaborde, A.: Inference of human beings’ emotional states from speech in human-robot interactions. Int. J. Soc. Robot. 7, 1–13 (2015)

    Article  Google Scholar 

  15. Doering, N., Poeschl, S., Gross, H.M., Bley, A., Martin, C., Boehme, H.J.: User-centered design and evaluation of a mobile shopping robot. Int. J. Soc. Robot. 7(2), 203–225 (2015)

    Article  Google Scholar 

  16. Dragan, A., Srinivasa, S.: Formalizing assistive teleoperation. In: Robotics: Science and Systems (2012)

    Google Scholar 

  17. Feil-Seifer, D., Mataric, M.J.: Human robot interaction. In: Encyclopedia of Complexity and Systems Science (2009)

    Google Scholar 

  18. Ferreira, J., Dias, J.: Attentional mechanisms for socially interactive robots - a survey. IEEE Trans. Auton. Ment. Dev. 6(2), 110–125 (2014)

    Article  Google Scholar 

  19. Fischinger, D., Einramhof, P., Papoutsakis, K., Wohlkinger, W., Mayer, P., Panek, P., Hofmann, S., Koertner, T., Weiss, A., Argyros, A., Vincze, M.: Hobbit, a care robot supporting independent living at home: first prototype and lessons learned. In: Robotics and Autonomous Systems (2014). In Press

    Google Scholar 

  20. Fong, T., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots. Robot. Auton. Syst. 42(3–4), 143–166 (2003)

    Article  MATH  Google Scholar 

  21. Goodrich, M.A., Schultz, A.C.: Human-robot interaction: a survey. Found. Trends Hum. Comput. Inter. 1(3), 203–275 (2007)

    Article  MATH  Google Scholar 

  22. Grigore, E., Eder, K., Pipe, A., Melhuish, C., Leonards, U.: Joint action understanding improves robot-to-human object handover. In: IEEE/RSJ IROS (2013)

    Google Scholar 

  23. Han, J., Gilmartin, E., Campbell, N.: Herme, yet another interactive conversational robot. In: Humaine Association Conference on Affective Computing and Intelligent Interaction (2013)

    Google Scholar 

  24. Hirose, M., Ogawa, K.: Honda humanoid robots development. Philos. Trans. Royal Soc. Lond. A: Math. Phys. Eng. Sci. 365(1850), 11–19 (2007)

    Article  Google Scholar 

  25. Hoai, M., De la Torre, F.: Max-margin early event detectors. IJCV 107(2), 191–202 (2014)

    Article  MathSciNet  Google Scholar 

  26. Jabon, M., Bailenson, J., Pontikakis, E., Takayama, L., Nass, C.: Facial expression analysis for predicting unsafe driving behavior. IEEE Pervasive Comput. 10(4), 84–95 (2011)

    Article  Google Scholar 

  27. Jones, C., Deeming, A.: Affective human-robotic interaction. In: Peter, C., Beale, R. (eds.) Affect and Emotion in HCL. LNCS, vol. 4868, pp. 175–185. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  28. Kaefer, P., Germino, K., Venske, D., Williams, A.: Computational awareness in a tactile-responsive humanoid robot comedian. In: IEEE International Conference on Systems, Man, and Cybernetics (2013)

    Google Scholar 

  29. Karg, M., Kirsch, A.: Acquisition and use of transferable, spatio-temporal plan representations for human-robot interaction. In: IEEE/RSJ IROS (2012)

    Google Scholar 

  30. Keizer, S., Kastoris, P., Foster, M., Deshmukh, A., Lemon, O.: Evaluating a social multi-user interaction model using a Nao robot. In: IEEE RO-MAN (2014)

    Google Scholar 

  31. Li, K., Xu, F., Wang, J., Dai, Q., Liu, Y.: A data-driven approach for facial expression synthesis in video. In: IEEE CVPR (2012)

    Google Scholar 

  32. Littlewort, G.C., Bartlett, M.S., Fasel, I.R., Chenu, J., Kanda, T., Ishiguro, H., Movellan, J.R.: Towards social robots: automatic evaluation of human-robot interaction by face detection and expression classification. In: NIPS (2004)

    Google Scholar 

  33. Liu, C., Ham, J., Postma, E., Midden, C., Joosten, B., Goudbeek, M.: Representing affective facial expressions for robots and embodied conversational agents by facial landmarks. Int. J. Soc. Robot. 5(4), 619–626 (2013)

    Article  Google Scholar 

  34. Liu, H., Sun, F.: Semi-supervised particle filter for visual tracking. In: IEEE ICRA (2009)

    Google Scholar 

  35. Malmir, M., Forster, D., Youngstrom, K., Morrison, L., Movellan, J.: Home alone: Social robots for digital ethnography of toddler behavior. In: IEEE ICCVW (2013)

    Google Scholar 

  36. McColl, D., Zhang, Z., Nejat, G.: Human body pose interpretation and classification for social human-robot interaction. Int. J. Soc. Robot. 3(3), 313–332 (2011)

    Article  Google Scholar 

  37. McKeague, S., Liu, J., Yang, G.-Z.: An asynchronous RGB-D sensor fusion framework using monte-carlo methods for hand tracking on a mobile robot in crowded environments. In: Herrmann, G., Pearson, M.J., Lenz, A., Bremner, P., Spiers, A., Leonards, U. (eds.) ICSR 2013. LNCS, vol. 8239, pp. 491–500. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  38. Menna, M., Gianni, M., Pirri, F.: Learning the dynamic process of inhibition and task switching in robotics cognitive control. In: ICMLA (2013)

    Google Scholar 

  39. Michel, D., Papoutsakis, K., Argyros, A.A.: Gesture recognition supporting the interaction of humans with socially assistive robots. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., McMahan, R., Jerald, J., Zhang, H., Drucker, S.M., Kambhamettu, C., El Choubassi, M., Deng, Z., Carlson, M. (eds.) ISVC 2014, Part I. LNCS, vol. 8887, pp. 793–804. Springer, Heidelberg (2014)

    Google Scholar 

  40. Nickel, K., Stiefelhagen, R.: Visual recognition of pointing gestures for human-robot interaction. Image Vis. Comput. 25(12), 1875–1884 (2007)

    Article  Google Scholar 

  41. Niculescu, A., van Dijk, B., Nijholt, A., Li, H., See, S.: Making social robots more attractive: the effects of voice pitch, humor and empathy. Int. J. Soc. Robot. 5(2), 171–191 (2013)

    Article  Google Scholar 

  42. Novikova, J., Watts, L.: Towards artificial emotions to assist social coordination in HRI. Int. J. Soc. Robot. 7(1), 77–88 (2015)

    Article  Google Scholar 

  43. Park, C., Lee, S.: Real-time 3D pointing gesture recognition for mobile robots with cascade HMM and particle filter. Image Vis. Comput. 29(1), 51–63 (2011)

    Article  Google Scholar 

  44. Park, I., Kim, J., Lee, J., Oh, J.: Mechanical design of humanoid robot platform KHR-3 (KAIST Humanoid Robot 3: HUBO). In: HUMANOIDS (2005)

    Google Scholar 

  45. Silvera-Tawil, D., Rye, D., Velonaki, M.: Interpretation of social touch on an artificial arm covered with an EIT-based sensitive skin. Int. J. Soc. Robot. 6(4), 489–505 (2014)

    Article  Google Scholar 

  46. Silvera-Tawil, D., Rye, D., Velonaki, M.: Artificial skin and tactile sensing for socially interactive robots: a review. Robot. Auton. Syst. 63(3), 230–243 (2015)

    Article  Google Scholar 

  47. Srinivasa, S., Ferguson, D., Helfrich, C., Berenson, D., Collet, A., Diankov, R., Gallagher, G., Hollinger, G., Kuffner, J., Weghe, M.V.: HERB: a home exploring robotic butler. Auton. Robot. 28(1), 5–20 (2010)

    Article  Google Scholar 

  48. Stiehl, W., Lieberman, J., Breazeal, C., Basel, L., Lalla, L., Wolf, M.: Design of a therapeutic robotic companion for relational, affective touch. In: IEEE RO-MAN (2005)

    Google Scholar 

  49. Strabala, K., Lee, M.K., Dragan, A., Forlizzi, J., Srinivasa, S.S.: Towards seamless human-robot handovers. J. Hum. Robot. Inter. 1(1), 1–23 (2013)

    Google Scholar 

  50. Trovato, G., Kishi, T., Endo, N., Zecca, M., Hashimoto, K., Takanishi, A.: Cross-cultural perspectives on emotion expressive humanoid robotic head: recognition of facial expressions and symbols. Int. J. Soc. Robot. 5(4), 515–527 (2013)

    Article  Google Scholar 

  51. Tsui, K., Flynn, K., McHugh, A., Yanco, H., Kontak, D.: Designing speech-based interfaces for telepresence robots for people with disabilities. In: ICORR (2013)

    Google Scholar 

  52. Viciana-Abad, R., Marfil, R., Perez-Lorenzo, J.M., Bandera, J.P., Romero-Garces, A., Reche-Lopez, P.: Audio-visual perception system for a humanoid robotic head. Sens. 14(6), 9522–9545 (2014)

    Article  Google Scholar 

  53. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE CVPR (2001)

    Google Scholar 

  54. Wang, Y., Cheng, H., Zheng, Y., Yang, L.: Face recognition in the wild by mining frequent feature itemset. In: Li, S., Liu, C., Wang, Y. (eds.) CCPR 2014, Part II. CCIS, vol. 484, pp. 331–340. Springer, Heidelberg (2014)

    Google Scholar 

  55. Wang, Z., Ambrus, R., Jensfelt, P., Folkesson, J.: Modeling motion patterns of dynamic objects by IOHMM. In: IEEE/RSJ IROS (2014)

    Google Scholar 

  56. Xiao, Y., Zhang, Z., Beck, A., Yuan, J., Thalmann, D.: Human-robot interaction by understanding upper body gestures. Presence 23(2), 133–154 (2014)

    Article  Google Scholar 

  57. Yan, H., Ang, M.H.J., Poo, A.N.: A survey on perception methods for human-robot interaction in social robots. Int. J. Soc. Robot. 6(1), 85–119 (2014)

    Article  Google Scholar 

  58. Yanik, P., Manganelli, J., Merino, J., Threatt, A., Brooks, J., Green, K., Walker, I.: Use of kinect depth data and growing neural gas for gesture based robot control. In: PervasiveHealth (2012)

    Google Scholar 

  59. Yin, X., Xie, M.: Finger identification and hand posture recognition for human-robot interaction. Image Vis. Comput. 25(8), 1291–1300 (2007)

    Article  Google Scholar 

  60. Zaraki, A., Mazzei, D., Giuliani, M., De Rossi, D.: Designing and evaluating a social gaze-control system for a humanoid robot. IEEE Trans. Hum. Mach. Syst. 44(2), 157–168 (2014)

    Article  Google Scholar 

Download references

Acknowledgment

This work was partially supported by NSFC (No.61305033, 61273256), Fundamental Research Funds for the Central Universities (ZYGX2013J088, ZYGX2014Z009) and SRF for ROCS, SEM

Author information

Authors and Affiliations

  1. Center for Robotics, School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China

    Lu Yang, Hong Cheng, Jiasheng Hao, Yanli Ji & Yiqun Kuang

Authors
  1. Lu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hong Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiasheng Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanli Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yiqun Kuang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hong Cheng .

Editor information

Editors and Affiliations

  1. Gwangju Institute of Science and Technology, Gwangju, Korea (Republic of)

    Yo-Sung Ho

  2. Chinese Academy of Sciences, Institute of Automation, Beijing, China

    Jitao Sang

  3. KAIST, Daejeon, Korea (Republic of)

    Yong Man Ro

  4. KAIST, Daejeon, Korea (Republic of)

    Junmo Kim

  5. College of Computer Science, Zhejiang University, Hangzhou, China

    Fei Wu

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Yang, L., Cheng, H., Hao, J., Ji, Y., Kuang, Y. (2015). A Survey on Media Interaction in Social Robotics. In: Ho, YS., Sang, J., Ro, Y., Kim, J., Wu, F. (eds) Advances in Multimedia Information Processing -- PCM 2015. PCM 2015. Lecture Notes in Computer Science(), vol 9315. Springer, Cham. https://doi.org/10.1007/978-3-319-24078-7_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-24078-7_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-24077-0

  • Online ISBN: 978-3-319-24078-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation