Proactive Intention Recognition for Joint Human-Robot Search and Rescue Missions Through Monte-Carlo Planning in POMDP Environments

  • Dimitri OgnibeneEmail author
  • Lorenzo Mirante
  • Letizia Marchegiani
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11876)


Proactively perceiving others’ intentions is a crucial skill to effectively interact in unstructured, dynamic and novel environments. This work proposes a first step towards embedding this skill in support robots for search and rescue missions. Predicting the responders’ intentions, indeed, will enable exploration approaches which will identify and prioritise areas that are more relevant for the responder and, thus, for the task, leading to the development of safer, more robust and efficient joint exploration strategies. More specifically, this paper presents an active intention recognition paradigm to perceive, even under sensory constraints, not only the target’s position but also the first responder’s movements, which can provide information on his/her intentions (e.g. reaching the position where he/she expects the target to be). This mechanism is implemented by employing an extension of Monte-Carlo-based planning techniques for partially observable environments, where the reward function is augmented with an entropy reduction bonus. We test in simulation several configurations of reward augmentation, both information theoretic and not, as well as belief state approximations and obtain substantial improvements over the basic approach.


Active vision Active perception Active intention recognition 


  1. 1.
    Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002). Scholar
  2. 2.
    Bacim, F., Ragan, E.D., Stinson, C., et al.: Collaborative navigation in virtual search and rescue. In: 2012 IEEE Symposium on 3D User Interfaces (3DUI), pp. 187–188. IEEE (2012)Google Scholar
  3. 3.
    Baker, C.L., Jara-Ettinger, J., Saxe, R., Tenenbaum, J.B.: Rational quantitative attribution of beliefs, desires and percepts in human mentalizing. Nat. Hum. Behav. 1(4), 0064 (2017)CrossRefGoogle Scholar
  4. 4.
    Beck, Z., Teacy, L., Rogers, A., Jennings, N.R.: In: AAMAS (2016)Google Scholar
  5. 5.
    Bellemare, M.G., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D., Munos, R.: Unifying count-based exploration and intrinsic motivation, June 2016. arXiv preprint arXiv:1606.01868
  6. 6.
    Boers, Y., Driessen, H., Bagchi, A., Mandal, P.: Particle filter based entropy. In: 2010 13th Conference on Information Fusion (FUSION), pp. 1–8. IEEE (2010)Google Scholar
  7. 7.
    de Croon, G.: Adaptive active vision. Ph.D. thesis, Universiteit Maastricht (2008)Google Scholar
  8. 8.
    Demiris, Y.: Prediction of intent in robotics and multi-agent systems. Cog. Proc. 8(3), 151–158 (2007)CrossRefGoogle Scholar
  9. 9.
    Denzler, J., Brown, C.: Information theoretic sensor data selection for active object recognition and state estimation. IEEE TPAMI 24(2), 145–157 (2002)CrossRefGoogle Scholar
  10. 10.
    Friston, K., Rigoli, F., Ognibene, D., Mathys, C., Fitzgerald, T., Pezzulo, G.: Active inference and epistemic value. Cogn. Neurosci. 6, 1–28 (2015)CrossRefGoogle Scholar
  11. 11.
    Goldhoorn, A., Garrell, A., Alquezar, R., Sanfeliu, A.: Continuous real time POMCP to find-and-follow people by a humanoid service robot. In: Humanoids 2014. IEEE (2014)Google Scholar
  12. 12.
    Kaelbling, L., Littman, M., Cassandra, A.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101(1), 99–134 (1998)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Kruijff, T., Linder, P., Gianni, P., Pizzoli, S., Pianese, C.: Rescue robots at earthquake-hit mirandola, italy: a field report. In: IEEE Safety, Security, and Rescue Robotics (2012)Google Scholar
  14. 14.
    Lauri, M., Ritala, R.: Planning for robotic exploration based on forward simulation. Robot. Auton. Syst. 83, 15–31 (2016)CrossRefGoogle Scholar
  15. 15.
    Lee, K., Ognibene, D., Chang, H.J., Kim, T.K., Demiris, Y.: Stare: spatio-temporal attention relocation for multiple structured activities detection. In: IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 5916–5927 (2015)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Mirolli, G.B.M. (ed.): Intrinsically Motivated Learning in Natural and Artificial Systems. Springer, Heidelberg (2014). Scholar
  17. 17.
    Ognibene, D., Baldassare, G.: Ecological active vision: four bioinspired principles to integrate bottom-up and adaptive top-down attention tested with a simple camera-arm robot. In: IEEE Transactions on Autonomous Mental Development, vol. 7, no. 1, pp. 3–25 (2015)CrossRefGoogle Scholar
  18. 18.
    Ognibene, D., Chinellato, E., Sarabia, M., Demiris, Y.: Contextual action recognition and target localization with an active allocation of attention on a humanoid robot. Bioinspiration Biomimetics 8(3), 035002 (2013)CrossRefGoogle Scholar
  19. 19.
    Ognibene, D., Demiris, Y.: Towards active event recognition. In: IJCAI, vol. 2013 (2013)Google Scholar
  20. 20.
    Ognibene, D., Giglia, G., Marchegiani, L., Rudrauf, D.: Implicit perception simplicity and explicit perception complexity in sensorimotor comunication. Phys. Life Rev. 28, 36–38 (2019)CrossRefGoogle Scholar
  21. 21.
    Paletta, L., Fritz, G., Seifert, C.: Q-learning of sequential attention for visual object recognition from informative local descriptors. In: Proceedings ICML 2005, p. 656 (2005)Google Scholar
  22. 22.
    Pezzulo, G., Donnarumma, F., Dindo, H., et al.: The body talks: sensorimotor communication and its brain and kinematic signatures. Phys. Life Rev. 28, 1–21 (2019)CrossRefGoogle Scholar
  23. 23.
    Ramirez, M., Geffner, H.: Goal recognition over POMDPs: Inferring the intention of a POMDP agent. In: IJCAI, Barcelona (2011)Google Scholar
  24. 24.
    Ramirez, M., Geffner, H.: Plan recognition as planning. In: IJCAI (2009)Google Scholar
  25. 25.
    Shah, S., Dey, D., Lovett, C., Kapoor, A.: AirSim: high-fidelity visual and physical simulation for autonomous vehicles. In: Field and Service Robotics (2017)Google Scholar
  26. 26.
    Silver, D., Veness, J.: Monte-Carlo planning in large POMDPs. In: 24th Advances in Neural Information Processing Systems, NIPS 2010, pp. 2164–2172 (2010)Google Scholar
  27. 27.
    Sprague, N., Ballard, D.: Eye movements for reward maximization. In: Advances in Neural Information Processing Systems 16, Cambridge (2004)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Computer Science and Electronic EngineeringUniversity of EssexColchesterUK
  2. 2.Department of Electronic SystemsAalborg UniversityAalborgDenmark

Personalised recommendations