Human-Robot Teaming: Approaches from Joint Action and Dynamical Systems

  • Tariq IqbalEmail author
  • Laurel D. Riek
Reference work entry


As robots start to work alongside people, they are expected to coordinate fluently with humans in teams. Many researchers have explored the problems involved in building more interactive and cooperative robots. In this chapter, we discuss recent work and the main application areas in human-robot teaming. We also shed light on some practical challenges to achieving fluent human-robot coordination and conclude the chapter with future directions for approaching these problems.


Human-robot interaction Human-robot teaming Joint action Dynamical group modeling Coordination 


  1. 1.
    H. Admoni, A. Dragan, S. Srinivasa, B. Scassellati, Deliberate delays during robot-to-human handovers improve compliance with gaze communication, in International Conference on Human-Robot Interaction, Bielefeld (2014)Google Scholar
  2. 2.
    H. B. Amor, G. Neumann, S. Kamthe, O. Kroemer, J. Peters, Interaction primitives for human-robot cooperation tasks, in IEEE Confernce on Robotics and Automation, Hong Kong (2014)Google Scholar
  3. 3.
    B. Argall, S. Chernova, M. Veloso, A survey of robots learning from demonstration. Robot. Auton. Syst. 57, 469–483 (2009)CrossRefGoogle Scholar
  4. 4.
    T. Brys, A. Harutyunyan, V.U. Brussel, M.E. Taylor, Reinforcement Learning from Demonstration through Shaping, in Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, Buenos Aires (2015)Google Scholar
  5. 5.
    M. Cakmak, S.S. Srinivasa, M.K. Lee, S. Kiesler, J. Forlizzi, Using spatial and temporal contrast for fluent robot-human hand-overs, in Proceedings of ACM/IEEE HRI, Lausanne (2011)Google Scholar
  6. 6.
    D. Chan, L.D. Riek, Object proposal algorithms in the wild: are they generalizable to robot perception? in Review (2017)Google Scholar
  7. 7.
    D. Chan, A. Taylor, L.D. Riek, Faster robot perception using salient depth perception, in IROS (2017)Google Scholar
  8. 8.
  9. 9.
    D.R. Forsyth, Group dynamics, 4th edn. (T. Wadsworth, 2009)Google Scholar
  10. 10.
    E.C. Grigore, K. Eder, A.G. Pipe, C. Melhuish, U. Leonards, Joint action understanding improves robot-to-human object handover, in IEEE/RSJ Internation Conference on Intelligent Robots and Systems (IROS), Tokyo (2013)Google Scholar
  11. 11.
    C.J. Hayes, M. Moosaei, L.D. Riek, Exploring implicit human responses to robot mistakes in a learning from demonstration task, in Robot and Human Interactive Communication (RO-MAN), New York (2016)Google Scholar
  12. 12.
    G. Hoffman, C. Breazeal, Collaboration in human-robot teams, in AIAA Intelligent Systems Technical Conference (2007)Google Scholar
  13. 13.
    G. Hoffman, C. Breazeal, Cost-based anticipatory action selection for human–robot fluency. IEEE Trans. Robot. 23(5), 952–961 (2007)CrossRefGoogle Scholar
  14. 14.
    G. Hoffman, C. Breazeal, Effects of anticipatory action on human-robot teamwork efficiency, fluency, and perception of team, in International Conference on Human-Robot Interaction, Arlington (2007)Google Scholar
  15. 15.
    G. Hoffman, C. Breazeal, Anticipatory perceptual simulation for human-robot joint practice: theory and application study, in AAAI, Chicago (2008)Google Scholar
  16. 16.
    G. Hoffman, W. Ju, Designing robots with movement in mind. J. Hum.-Robot Interact 3(1), 89–122 (2014)CrossRefGoogle Scholar
  17. 17.
    G. Hoffman, G. Weinberg, Synchronization in human-robot Musicianship, in International Symposium on Robot and Human Interactive Communication, Viareggio (2010)Google Scholar
  18. 18.
    G. Hoffman, G. Weinberg, Interactive improvisation with a robotic marimba player in Musical Robots and Interactive Multimodal Systems (Springer, 2011), pp. 233–251Google Scholar
  19. 19.
    C.M. Huang, M. Cakmak, B. Mutlu, Adaptive coordination strategies for human-robot handovers, in Robotics: Science and Systems (IEEE Robotics and Automation Society, 2015)Google Scholar
  20. 20.
    T. Iqbal, L. Riek, Assessing group synchrony during a rhythmic social activity: a systemic approach, in Proceedings of the Conference of the International Society for Gesture Studies (ISGS), San Diego (2014)Google Scholar
  21. 21.
    T. Iqbal, L.D. Riek, Role distribution in synchronous human-robot joint action, in Proceedings of IEEE RO-MAN, Towards a Framework for Joint Action, Portland (2014)Google Scholar
  22. 22.
    T. Iqbal, L.D. Riek, A method for automatic detection of psychomotor entrainment. IEEE Trans. Affect. Comput. 7(1), 3–16 (2016)Google Scholar
  23. 23.
    T. Iqbal, L.D. Riek, Detecting and synthesizing synchronous joint action in human-robot teams, in: International Conference on Multimodal Interaction, Seattle (2015)Google Scholar
  24. 24.
    T. Iqbal, L.D. Riek, Coordination dynamics in multi-human multi-robot teams. IEEE Robot. Autom. Letters (RA-L) 2(3), 1712–1717 (2017)CrossRefGoogle Scholar
  25. 25.
    T. Iqbal, M.J. Gonzales, L.D. Riek, A model for time-synchronized sensing and motion to support human-robot fluency, in ACM/IEEE Human-Robot Interaction, Workshop on Timing in HRI, Bielefeld (2014)Google Scholar
  26. 26.
    T. Iqbal, M.J. Gonzales, L.D. Riek, Mobile robots and marching humans: measuring synchronous joint action while in motion, in AAAI Fall Symposium on AI-HRI, Arlington (2014)Google Scholar
  27. 27.
    T. Iqbal, M.J. Gonzales, L.D. Riek, Joint action perception to enable fluent human-robot teamwork, in Proceedings of IEEE Robot and Human Interactive Communication, Kobe (2015)Google Scholar
  28. 28.
    T. Iqbal, M. Moosaei, L.D. Riek, Tempo adaptation and anticipation methods for human-robot teams, in Robotics: Science and Systems, Planning for HRI: Shared Autonomy and Collaborative Robotics Workshop, Ann Arbor (2016).Google Scholar
  29. 29.
    T. Iqbal, S. Rack, L.D. Riek, Movement coordination in human-robot teams: a dynamical systems approach. IEEE Trans. Robot. 32(4), 909–919 (2016)CrossRefGoogle Scholar
  30. 30.
    N. Jarrassé, T. Charalambous, E. Burdet, A framework to describe, analyze and generate interactive motor behaviors. PLoS One 7(11), e49945 (2012)CrossRefGoogle Scholar
  31. 31.
  32. 32.
    G. Knoblich, J.S. Jordan, Action coordination in groups and individuals: learning anticipatory control. J. Exp. Psychol. Learn. Mem. Cogn. 29(5), 1006–1016 (2003)CrossRefGoogle Scholar
  33. 33.
    G. Knoblich, S. Butterfill, N. Sebanz, Psychological research on joint action: theory and data, in Psychology of Learning and Motivation. Advances in Research and Theory. 54 (2011), p. 59Google Scholar
  34. 34.
    I. Konvalinka, P. Vuust, A. Roepstorff, C.D. Frith, Follow you, follow me: continuous mutual prediction and adaptation in joint tapping. Q. J. Exp. Psychol. 63(11), 2220–2230 (2010)CrossRefGoogle Scholar
  35. 35.
    I. Konvalinka, D. Xygalatas, J. Bulbulia, U. Schjødt, E.M. Jegindø, S. Wallot, G. Van Orden, A. Roepstorff, Synchronized arousal between performers and related spectators in a fire-walking ritual. Proc. Natl. Acad. Sci. USA 108(20), 8514–8519 (2011)CrossRefGoogle Scholar
  36. 36.
    H.S. Koppula, A. Jain, A. Saxena, Anticipatory planning for human-robot teams. Springer Tracts Adv Robot 109, 453–470 (2016)CrossRefGoogle Scholar
  37. 37.
    I. Leite, G. Castellano, A. Pereira, C. Martinho, A. Paiva, Modelling empathic behaviour in a robotic game companion for children: an ethnographic study in real-world settings, in ACM/IEEE International Conference on Human-Robot Interaction, Boston (2012)Google Scholar
  38. 38.
    T. Lorenz, A. Mortl, B. Vlaskamp, A. Schubo, S. Hirche, Synchronization in a goal-directed task: human movement coordination with each other and robotic partners, in Proc IEEE RO-MAN, Atlanta (2011)Google Scholar
  39. 39.
    J. Mainprice, D. Berenson, Human-robot collaborative manipulation planning using early prediction of human motion, in IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo (2013)Google Scholar
  40. 40.
    A. Moon, D.M. Troniak, B. Gleeson, M.K.X.J. Pan, M. Zheng, B.A. Blumer, K. MacLean, E.A. Croft, Meet me where I’m gazing: how shared attention gaze affects human-robot handover timing, in ACM/IEEE International Conference Human-Robot Interaction, Bielefeld (2014)Google Scholar
  41. 41.
    A. Mörtl, T. Lorenz, S. Hirche, Rhythm patterns interaction synchronization behavior for human-robot joint action. PloS One 9(4), e95195 (2014)CrossRefGoogle Scholar
  42. 42.
  43. 43.
    Z. Néda, E. Ravasz, Y. Brechet, T. Vicsek, A.L. Barabási, Self-organizing processes: the sound of many hands clapping. Nature 403(6772), 849–850 (2000)CrossRefGoogle Scholar
  44. 44.
    R.D. Newman-Norlund, M.L. Noordzij, R.G.J. Meulenbroek, H. Bekkering, Exploring the brain basis of joint action: co-ordination of actions, goals and intentions. Soc Neurosci 2(1), 48–65 (2007)CrossRefGoogle Scholar
  45. 45.
    S. Niekum, S. Chitta, Incremental semantically grounded learning from demonstration. Robot. Sci. Syst. IX (2013)Google Scholar
  46. 46.
    A. Nigam, L.D. Riek, Social context perception for mobile robots, in IEEE/RSJ Intelligent Robots and Systems (IROS), Hamburg (2015)Google Scholar
  47. 47.
    S. Nikolaidis, K. Gu, R. Ramakrishnan, J. Shah, R.O. May, Efficient model learning for human-robot collaborative tasks (2014), pp. 1–9.
  48. 48.
    G. Novembre, L.F. Ticini, S. Schütz-Bosbach, P.E. Keller, Motor simulation and the coordination of self and other in real-time joint action. Soc. Cogn. Affect. Neurosci. 9(8), 1062–1068 (2014)CrossRefGoogle Scholar
  49. 49.
    M.F. O’Connor, L.D. Riek, Detecting social context: a method for social event classification using naturalistic multimodal data, in Automatic Face and Gesture Recognition (FG), Ljubljana (2015)Google Scholar
  50. 50.
    K. Ong, G. Seet, S. Sim, An implementation of seamless human-robot interaction for telerobotics. Int. J. Adv. Robot. Syst. 5(2), 18 (2008)CrossRefGoogle Scholar
  51. 51.
    Open Source Robotics Foundation, (2017),
  52. 52.
    C. Pérez-DArpino, J. Shah, Fast target prediction of human reaching motion for cooperative human-robot manipulation tasks using time series classification, in International Conference on Robotics and Automation, Seattle (2015)Google Scholar
  53. 53.
    R. Poppe, A survey on vision-based human action recognition. Image Vis. Comput. 28(6), 976–990 (2010)CrossRefGoogle Scholar
  54. 54.
  55. 55.
    S. Rack, T. Iqbal, L. Riek, Enabling synchronous joint action in human-robot teams, in Proceedings of ACM/IEEE Human-Robot Interaction, Portland (2015)Google Scholar
  56. 56.
    M. Ramanathan, Y. Wy, E.K. Teoh, Human action recognition with video data : research and evaluation challenges. IEEE Trans. Hum.-Mach. Syst. 44(5), 650–663 (2014)CrossRefGoogle Scholar
  57. 57.
    M.J. Richardson, R.L. Garcia, T.D. Frank, M. Gergor, K.L. Marsh, Measuring group synchrony: a cluster-phase method for analyzing multivariate movement time-series. Front. Physiol. 3, 405 (2012)CrossRefGoogle Scholar
  58. 58.
    L.D. Riek, The social co-robotics problem space: six key challenges, in Proceedings of RSS, Robotics Challenges and Visions (2013)Google Scholar
  59. 59.
    L.D. Riek, P. Robinson, Real-time empathy: Facial mimicry on a robot, in International Conference on Multimodal Interfaces, Affective Interaction in Natural Environments (AFFINE), Chania (2008)Google Scholar
  60. 60.
    L.D. Riek, T.C. Rabinowitch, P. Bremner, A. Pipe, M. Fraser, P. Robinson, Cooperative gestures: effective signaling for humanoid robots, in ACM/IEEE International Conference on Human-Robot Interaction, Osaka (2010)Google Scholar
  61. 61.
    L.D. Riek, P.C. Paul, P. Robinson, When my robot smiles at me: Enabling human-robot rapport via real-time head gesture mimicry. J. Multimodal User Interfaces 3, 99–108 (2010)CrossRefGoogle Scholar
  62. 62.
    L.D. Riek, T.C. Rabinowitch, P. Bremner, A.G. Pipe, M. Fraser, P. Robinson, Cooperative gestures: effective signaling for humanoid robots, in Proceedings of ACM/IEEE HRI, Osaka (2010)Google Scholar
  63. 63.
    M.S. Ryoo, L. Matthies, First-person activity recognition: what are they doing to me? in Proceedings of IEEE Computer Vision and Pattern Recognition, Portland (2013)Google Scholar
  64. 64.
    M.S. Ryoo, T.J. Fuchs, L. Xia, J. Aggarwal, L. Matthies, Robot-centric activity prediction from first-person videos: what will they do to me? in Proceedings of ACM/IEEE HRI, Portland (2015)Google Scholar
  65. 65.
    N. Sebanz, G. Knoblich, Prediction in joint action: what, when, and where. Top. Cogn. Sci. 1(2), 353–367 (2009)CrossRefGoogle Scholar
  66. 66.
    N. Sebanz, H. Bekkering, G. Knoblich, Joint action: bodies and minds moving together. Trends Cogn. Sci. 10(2), 70–76 (2006)CrossRefGoogle Scholar
  67. 67.
    J. Shah, C. Breazeal, An Empirical Analysis of Team Coordination Behaviors and Action Planning With Application to Human-Robot Teaming. Hum. Factors J. Hum. Factors Ergon. Soc. 52(2), 234 (2010)CrossRefGoogle Scholar
  68. 68.
    J. Shah, J. Wiken, B. Williams, C. Breazeal, Improved human-robot team performance using Chaski, a human-inspired plan execution system, in Proceedings of 6th International Conference on Human-Robot Interaction, Lausanne (2011)Google Scholar
  69. 69.
    C. Shi, M. Shiomi, C. Smith, T. Kanda, H. Ishiguro A model of distributional handing interaction for a mobile robot in Robotics: Science and Systems (2013), pp. 24–28Google Scholar
  70. 70.
    P. Słowiński, C. Zhai, F. Alderisio, R. Salesse, M. Gueugnon, L. Marin, B.G. Bardy, M. di Bernardo, K. Tsaneva-Atanasova, Dynamic similarity promotes interpersonal coordination in joint-action. J. R. Soc. Interface 13(116), 20151093 (2016)Google Scholar
  71. 71.
    K. Strabala, M.K. Lee, A. Dragan, J. Forlizzi, S.S. Srinavasa, M. Cakmak, V. Micelli, Towards seamless human-robot handovers. J. Hum.-Robot Interact. 2(1), 112–132 (2013)CrossRefGoogle Scholar
  72. 72.
    A. Thomaz, G. Hoffman, M. Cakmak, Computational human-robot interaction. Found. Trends Robot. 4(2–3), 105–223 (2016)CrossRefGoogle Scholar
  73. 73.
    V.V. Unhelkar, C. Pérez-DArpino, L. Stirling, J. Shah, Human-robot co-navigation using anticipatory indicators of human walking motion, in IEEE Conference on Robotics and Automation, Seattle (2015)Google Scholar
  74. 74.
    P. Valdesolo, J. Ouyang, D. DeSteno, The rhythm of joint action: synchrony promotes cooperative ability. J. Exp. Soc. Psychol. 46(4), 693–695 (2010)CrossRefGoogle Scholar
  75. 75.
    G. Varni, G. Volpe, A. Camurri, A system for real-time multimodal analysis of nonverbal affective social interaction in user-centric media. IEEE Trans. Multimedia 12(6), 576–590 (2010)CrossRefGoogle Scholar
  76. 76.
    C. Vesper, S. Butterfill, G. Knoblich, N. Sebanz, A minimal architecture for joint action. Neural Netw. 23(8–9), 998–1003 (2010)CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringUniversity of California San DiegoLa JollaUSA

Personalised recommendations