Abstract
In many realistic planning situations, any policy has a non-zero probability of reaching a dead-end. In such cases, a popular approach is to plan to maximize the probability of reaching the goal. While this strategy increases the robustness and expected autonomy of the robot, it considers that the robot gives up on the task whenever a dead-end is encountered. In this work, we consider planning for agents that pro-actively and autonomously resort to human help when an unavoidable dead-end is encountered (the so-called symbiotic agents). To this end, we develop a new class of Goal-Oriented Markov Decision Process that includes a set of human actions that ensures the existence of a proper policy, one that possibly resorts to human help. We discuss two different optimization criteria: minimizing the probability to use human help and minimizing the expected cumulative cost with a finite penalty for using human help for the first time. We show that for a large enough penalty both criteria are equivalent. We report on experiments with standard probabilistic planning domains for reasonably large problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We implicitly assume that every state has at least one applicable action.
References
Armstrong Crews, N., Veloso, M.: Oracular partially observable markov decision processes: a very special case. In: Proceedings of the IEEE ICRA (2007)
Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957)
Bertsekas, D.P., Tsitsiklis, J.N.: An analysis of stochastic shortest path problems. Math. Oper. Res. 16(3), 580–595 (1991). INFORMS
Bonet, B.: Labeled RTDP: improving the convergence of real-time dynamic programming. In: Proceedings ICAPS-03 (2003)
Bonet, B., Geffner, H.: Faster heuristic search algorithms for planning with uncertainty and full feedback. In: Proceedings of the IJCAI (2003)
Bonet, B., Geffner, H.: mGPT: a probabilistic planner based on heuristic search. J. Artif. Intell. Res. 24, 933–944 (2005)
Göbelbecker, M., Keller, T., Eyerich, P., Brenner, M., Nebel, B.: Coming up with good excuses: what to do when no plan can be found. In: ICAPS (2010)
Helmert, M.: The fast downward planning system. J. Artif. Intell. Res. 26, 191–246 (2006)
Karami, A.B., Jeanpierre, L., Mouaddib, A.I.: Partially observable markov decision process for managing robot collaboration with human. In: Proceedings of the 21st IEEE ICTAI (2009)
Kolobov, A., Daniel, M., Weld, S., Geffner, H.: Heuristic search for generalized stochastic shortest path MDPs. In: Proceedings of the ICAPS (2011)
Kolobov, A., Mausam, M., Weld, D.: A theory of goal-oriented MDPs with dead ends. In: Proceedings of the 28th Conference on UAI (2012)
Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley (2014)
Rosenthal, S., Biswas, J., Veloso, M.: An effective personal mobile robot agent through symbiotic human-robot interaction. In: Proceedings of the AAMAS (2010)
Rosenthal, S., Veloso, M., Dey, A.K.: Learning accuracy and availability of humans who help mobile robots. In: Proceedings of the AAAI (2011)
Schmidt-Rohr, S.R., Knoop, S., Lösch, M., Dillmann, R.: Reasoning for a multi-modal service robot considering uncertainty in human-robot interaction. In: Proceedings of the 3rd HRI (2008)
Steinmetz, M., Hoffmann, J., Buffet, O.: Revisiting goal probability analysis in probabilistic planning. In: Proceedings of the 26th ICAPS (2016)
Teichteil-Königsbuch, F.: Stochastic safest and shortest path problems. In: Proceedings of the NCAI (2012)
Trevizan, F., Teichteil-Königsbuch, F., Thiébaux, S.: Efficient solutions for stochastic shortest path problems with dead ends. In: Proceedings of 33rd Conference on UAI (2017)
Younes, H.L., Littman, M.L.: PPDDL1.0: an extension to PDDL for expressing planning domains with probabilistic effects. Technical report CMU-CS-04-162 (2004)
Acknowledgments
Authors received financial support from CAPES, FAPESP (grants #2015/01587-0 and #2016/01055-1) and CNPq (grants #303920/2016-5 and #420669/2016-7).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Andrés, I., de Barros, L.N., Mauá, D.D., Simão, T.D. (2018). When a Robot Reaches Out for Human Help. In: Simari, G., Fermé, E., Gutiérrez Segura, F., Rodríguez Melquiades, J. (eds) Advances in Artificial Intelligence - IBERAMIA 2018. IBERAMIA 2018. Lecture Notes in Computer Science(), vol 11238. Springer, Cham. https://doi.org/10.1007/978-3-030-03928-8_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-03928-8_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03927-1
Online ISBN: 978-3-030-03928-8
eBook Packages: Computer ScienceComputer Science (R0)