Deep Representations for Collaborative Robotics

  • Luis J. Manso
  • Pablo Bustos
  • Juan P. Bandera
  • Adrián Romero-Garcés
  • Luis V. Calderita
  • Rebeca Marfil
  • Antonio BanderaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10087)


Collaboration is an essential feature of human social interaction. Briefly, when two or more people agree on a common goal and a joint intention to reach that goal, they have to coordinate their actions to engage in joint actions, planning their courses of actions according to the actions of the other partners. The same holds for teams where the partners are people and robots, resulting on a collection of technical questions difficult to answer. Human-robot collaboration requires the robot to coordinate its behavior to the behaviors of the humans at different levels, e.g., the semantic level, the level of the content and behavior selection in the interaction, and low-level aspects such as the temporal dynamics of the interaction. This forces the robot to internalize information about the motions, actions and intentions of the rest of partners, and about the state of the environment. Furthermore, collaborative robots should select their actions taking into account additional human-aware factors such as safety, reliability and comfort. Current cognitive systems are usually limited in this respect as they lack the rich dynamic representations and the flexible human-aware planning capabilities needed to succeed in tomorrow human-robot collaboration tasks. Within this paper, we provide a tool for addressing this problem by using the notion of deep hybrid representations and the facilities that this common state representation offers for the tight coupling of planners on different layers of abstraction. Deep hybrid representations encode the robot and environment state, but also a robot-centric perspective of the partners taking part in the joint activity.


Deep representations Cognitive robots Agent-based robotic architecture 



This paper has been partially supported by the Spanish Ministerio de Economía y Competitividad TIN2015-65686-C5 and FEDER funds, and by the Innterconecta Programme 2011 project ITC-20111030 ADAPTA.


  1. 1.
  2. 2.
    Hayes-Roth, B.: A blackboard architecture for control. Artif. Intell. 26(3), 251–321 (1985)CrossRefGoogle Scholar
  3. 3.
    Blumenthal, S., Bruyninckx, H., Nowak, W., Prassler, E.: A scene graph based shared 3D world model for robotic applications. In: 2013 IEEE International Conference on Robotics and Automation, pp. 453–460, May 2013.
  4. 4.
    Brugali, D., Scandurra, P.: Component-based robotic engineering (Part I). IEEE Robot. Autom. Mag. 16, 84–96 (2009)CrossRefGoogle Scholar
  5. 5.
    Brugali, D., Shakhimardanov, A.: Component-based robotic engineering (Part II): systems and models. IEEE Robot. Autom. Mag. 17(1), 100–112 (2010)CrossRefGoogle Scholar
  6. 6.
    Calderita, L.V., Bustos, P., Suárez Mejías, C., Fernández, F., Bandera, A.: THERAPIST: towards an autonomous socially interactive robot for motor and neurorehabilitation therapies for children. In: 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops, vol. 1, pp. 374–377 (2013).
  7. 7.
    Corkill, D.D.: Blackboard systems. AI Expert 6(9), 40–47 (1991)Google Scholar
  8. 8.
    Erman, L.D., Hayes-Roth, F., Lesser, V.R., Reddy, D.R.: The Hearsay-II speech-understanding system: integrating knowledge to resolve uncertainty. ACM Comput. Surv. 12(2), 213–253 (1980)CrossRefGoogle Scholar
  9. 9.
    Foote, T.: tf: The transform library. In: 2013 IEEE International Conference on Technologies for Practical Robot Applications (TePRA), pp. 1–6. Open-Source Software workshop (2013)Google Scholar
  10. 10.
    Franklin, S., Graesser, A.: Is It an agent, or just a program?: A taxonomy for autonomous agents. In: Müller, J.P., Wooldridge, M.J., Jennings, N.R. (eds.) ATAL 1996. LNCS, vol. 1193, pp. 21–35. Springer, Heidelberg (1997). doi: 10.1007/BFb0013570 CrossRefGoogle Scholar
  11. 11.
    Gratal, X., Romero, J., Bohg, J., Kragic, D.: Visual servoing on unknown objects. Mechatronics 22(4), 423–435 (2012). CrossRefGoogle Scholar
  12. 12.
    Hutchinson, S., Hager, G., Corke, P.: A tutorial on visual servo control. IEEE Trans. Robot. Autom. 12(5), 651–670 (1996). CrossRefGoogle Scholar
  13. 13.
    Maes, P.: The agent network architecture (ANA). SIGART Bull. 2(4), 115–120 (1991)CrossRefGoogle Scholar
  14. 14.
    Manso, L.J.: Perception as stochastic sampling on dynamic graph spaces. Ph.D. thesis, Universidad de Extremadura (2013)Google Scholar
  15. 15.
    Manso, L., Bachiller, P., Bustos, P., Calderita, L.: RoboComp: a tool-based robotics framework. In: SIMPAR, Second International COnference on Simulation, Modelling and programming for Autonomous Robots (2010)Google Scholar
  16. 16.
    McManus, J.W.: Design and analysis of concurrent blackboard systems. Ph.D. thesis, College of William and Mary, Virginia (1992)Google Scholar
  17. 17.
    Minsky, M.: The Society of Mind. Simon & Schuster, New York (1988)Google Scholar
  18. 18.
    Newell, A.: Some problems of basic organization in problem-solving programs. Technical report, RAND Corporation (1962)Google Scholar
  19. 19.
    Oliver, G.S.: Pandamonium: a paradigm for learning. In: Proceedings of the Symposium on the Mechanization of Thought Processes, pp. 511–529 (1959)Google Scholar
  20. 20.
    Bustos, P., Martnez-Gomez, J., Garca-Varea, I., Rodrguez-Ruiz, L., Bachiller, P., Calderita, L., Manso, L.J., Sanchez, A., Bandera, A., Bandera, J.: Multimodal interaction with loki. In: Workshop of Physical Agents, Leon, Spain, pp. 1–8 (2013).
  21. 21.
    Poole, D., Mackworth, A.: Artificial Intelligence: Foundations of Computational Agents. Cambridge University Press, Cambridge (2010).
  22. 22.
    Rao, A.S., Georgeff, M.P.: BDI agents: from theory to practice. In: ICMAS, pp. 312–319 (1995)Google Scholar
  23. 23.
    Rusell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 3rd edn. Pearson, Upper Saddle River (2009)Google Scholar
  24. 24.
    Sloman, A., Wyatt, J., Hawes, N., Chappell, J., Kruijff, G.J.: Long term requirements for cognitive robotics. In: Cognitive Robotics Papers from the 2006 AAAI Workshop Technical Report WS0603, pp. 143–150. No. McCarthy (2006).
  25. 25.
    Wang, A.J.A., Qian, K.: Component-Oriented Programming. Wiley, Hoboken (2005)CrossRefGoogle Scholar
  26. 26.
    Kunze, L., Dolha, M.E., Guzman, E., Beetz, M.: Simulation-based temporal projection of everyday robot object manipulation. In: Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2011) (2011)Google Scholar
  27. 27.
    Clark, A.: An embodied cognitive science? Trends Cogn. Sci. 3(9), 345–351 (1999)CrossRefGoogle Scholar
  28. 28.
    Jelinek, F.: Statistical Methods for Speech Recognition. MIT Press, Cambridge (1997)Google Scholar
  29. 29.
    Calderita, L.V., Manso, L.J., Bustos, P., Suárez-Mejías, C., Fernández, F., Bandera, A.: THERAPIST: towards an autonomous socially interactive robot for motor and neurorehabilitation therapies for children. In: JMIR Rehabil Assist Technol (2014)Google Scholar
  30. 30.
    Romero-Garcés, A., Calderita, L.V., González, J., Bandera, J.P., Marfil, R., Manso, L.J., Bandera, A., Bustos, P.: Testing a fully autonomous robotic salesman in real scenarios. In: Conference: IEEE International Conference on Autonomous Robot Systems and Competitions (2015)Google Scholar
  31. 31.
    Manso, L.J.: Perception as stochastic sampling on dynamic graph spaces. Ph.D. dissertation, University of Extremadura, Spain (2013)Google Scholar
  32. 32.
    Kirsch, A., Kruse, T., Msenlechner, L.: An integrated planning and learning framework for human-robot interaction. In: 4th Workshop on Planning and Plan Execution for Real-World Systems (held in conjunction with ICApPS 09) (2009)Google Scholar
  33. 33.
    Beetz, M., Jain, D., Msenlechner, L., Tenorth, M.: Towards performing everyday manipulation activities. Robot. Autonom. Syst. 58, 1085–1095 (2010)CrossRefGoogle Scholar
  34. 34.
    Alami, R., Chatila, R., Clodic, A., Fleury, S., Herrb, M., Montreuil, V., Sisbot, E.A.: Towards human-aware cognitive robots. In: AAAI-06, Stanford Spring Symposium (2006)Google Scholar
  35. 35.
    Wintermute, S.: Imagery in cognitive architecture: representation and control at multiple levels of abstraction. Cogn. Syst. Res. 1920, 129 (2012)Google Scholar
  36. 36.
    Ali, M.: Contribution to decisional human-robot interaction: towards collaborative robot companions, Ph.D. thesis, Institut National de Sciences Appliquées de Toulouse, France (2012)Google Scholar
  37. 37.
    Holland, O.: The future of embodied artificial intelligence: machine consciousness? In: Iida, F., Pfeifer, R., Steels, L., Kuniyoshi, Y. (eds.) Embodied Artificial Intelligence. LNCS (LNAI), vol. 3139, pp. 37–53. Springer, Heidelberg (2004). doi: 10.1007/978-3-540-27833-7_3 CrossRefGoogle Scholar
  38. 38.
    Manso, L.J.: Perception as stochastic sampling on dynamic graph spaces. Ph.D. thesis, University of Extremadura, Spain (2013)Google Scholar
  39. 39.
  40. 40.

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Luis J. Manso
    • 1
  • Pablo Bustos
    • 1
  • Juan P. Bandera
    • 2
  • Adrián Romero-Garcés
    • 2
  • Luis V. Calderita
    • 1
    • 2
  • Rebeca Marfil
    • 2
  • Antonio Bandera
    • 2
    Email author
  1. 1.RoboLab GroupUniversity of ExtremaduraCáceresSpain
  2. 2.Dept. Tecnología ElectrónicaUniversity of MalagaMálagaSpain

Personalised recommendations