Abstract
A decidable logic is presented, in which queries can be posed about (i) the degree of belief in a propositional sentence after an arbitrary finite number of actions and observations and (ii) the utility of a finite sequence of actions after a number of actions and observations. The main contribution of this work is that a POMDP model specification is allowed to be partial or incomplete with no restriction on the lack of information specified for the model. The model may even contain information about non-initial beliefs. Essentially, entailment of arbitrary queries (expressible in the language) can be answered. A sound, complete and terminating decision procedure is provided.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
By “utility”, we mean ‘expected rewards’.
- 2.
[0,1] denotes \(\mathbb {R}\cap [0,1]\).
- 3.
Either the action is executable and there is a probability distribution (the summation is 1) or the action is inexecutable (the summation is 0). Letting the sum equal a number not 1 or 0 would lead to badly defined semantics.
- 4.
Inexecutability axioms are also called condition closure axioms.
- 5.
Probabilities used for specifying the initial belief-state are assumed given by a knowledge engineer or computed in an earlier process.
References
Smallwood, R., Sondik, E.: The optimal control of partially observable Markov processes over a finite horizon. Oper. Res. 21, 1071–1088 (1973)
Monahan, G.: A survey of partially observable Markov decision processes: theory, models, and algorithms. Manage. Sci. 28, 1–16 (1982)
Boutilier, C., Poole, D.: Computing optimal policies for partially observable decision processes using compact representations. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence (AAAI 1996), pp. 1168–1175. AAAI Press, Menlo Park (1996)
Geffner, H., Bonet, B.: High-level planning and control with incomplete information using POMDPs. In: Proceedings of the Fall AAAI Symposium on Cognitive Robotics, pp. 113–120. AAAI Press, Seattle (1998)
Hansen, E., Feng, Z.: Dynamic programming for POMDPs using a factored state representation. In: Proceedings of the Fifth International Conference on Artificial Intelligence, Planning and Scheduling (AIPS 2000), pp. 130–139 (2000)
Wang, C., Schmolze, J.: Planning with POMDPs using a compact, logic-based representation. In: Proceedings of the Seventeenth IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2005), pp. 523–530. IEEE Computer Society, Los Alamitos (2005)
Sanner, S., Kersting, K.: Symbolic dynamic programming for first-order POMDPs. In: Proceedings of the Twenty-Fourth National Conference on Artificial Intelligence (AAAI 2010), pp. 1140–1146. AAAI Press (2010)
Lison, P.: Towards relational POMDPs for adaptive dialogue management. In: Proceedings of the ACL 2010 Student Research Workshop, ACLstudent 2010, pp. 7–12. Association for Computational Linguistics, Stroudsburg (2010)
Wang, C., Khardon, R.: Relational partially observable MDPs. In: Fox, M., Poole, D. (eds.) Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2010). AAAI Press, Atlanta (2010)
De Weerdt, M., De Boer, F., Van der Hoek, W., Meyer, J.J.: Imprecise observations of mobile robots specified by a modal logic. In: Proceedings of the Fifth Annual Conference of the Advanced School for Computing and Imaging (ASCI 1999), pp. 184–190 (1999)
Bacchus, F., Halpern, J., Levesque, H.: Reasoning about noisy sensors and effectors in the situation calculus. Artif. Intell. 111, 171–208 (1999)
McCarthy, J.: Situations, actions and causal laws. Technical report, Stanford University (1963)
Gabaldon, A., Lakemeyer, G.: \({\cal ESP}\): A logic of only-knowing, noisy sensing and acting. In: Proceedings of the Twenty-Second National Conference on Artificial Intelligence (AAAI 2007), pp. 974–979. AAAI Press (2007)
Levesque, H., Lakemeyer, G.: Situations, si! Situation terms no! In: Proceedings of the Conference on Principles of Knowledge Representation and Reasoning (KR 2004), pp. 516–526. AAAI Press (2004)
Poole, D.: Decision theory, the situation calculus and conditional plans. Linköping Electron. Art. Comput. Inf. Sci. 8, 34 (1998)
Iocchi, L., Lukasiewicz, T., Nardi, D., Rosati, R.: Reasoning about actions with sensing under qualitative and probabilistic uncertainty. ACM Trans. Comput. Logic 10, 5:1–5:41 (2009)
Kwiatkowska, M., Norman, G., Parker, D.: Advances and challenges of probabilistic model checking. In: Proceedings of the Forty-Eighth Annual Allerton Conference on Communication, Control and Computing, pp. 1691–1698. IEEE Press (2010)
Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Aspects Comput. 6, 512–535 (1994)
Ross, S., Pineau, J., Chaib-draa, B., Kreitmann, P.: A bayesian approach for learning and planning in partially observable markov decision processes. J. Mach. Learn. Res. 12, 1729–1770 (2011)
Shirazi, A., Amir, E.: First-order logical filtering. Artif. Intell. 175, 193–219 (2011)
Rens, G., Meyer, T., Lakemeyer, G.: SLAP: Specification logic of actions with probability. J. Appl. Logic 12, 128–150 (2014)
Rens, G., Meyer, T., Lakemeyer, G.: A logic for specifying stochastic actions and observations. In: Beierle, C., Meghini, C. (eds.) FoIKS 2014. LNCS, vol. 8367, pp. 305–323. Springer, Heidelberg (2014)
Rens, G.: Formalisms for Agents Reasoning with Stochastic Actions and Perceptions. Ph.D. thesis, School of Mathematics, Statistics and Computer Science, University of KwaZulu-Natal (2014)
McCarthy, J., Hayes, P.: Some philosophical problems from the standpoint of artificial intelligence. Mach. Intell. 4, 463–502 (1969)
Rens, G., Meyer, T., Lakemeyer, G.: On the logical specification of probabilistic transition models. In: Proceedings of the Eleventh International Symposium on Logical Formalizations of Commonsense Reasoning (COMMONSENSE 2013), University of Technology, Sydney. UTSe Press (2013)
Saad, E.: Probabilistic reasoning by SAT solvers. In: Sossai, C., Chemello, G. (eds.) ECSQARU 2009. LNCS, vol. 5590, pp. 663–675. Springer, Heidelberg (2009)
Wang, C., Joshi, S., Khardon, R.: First order decision diagrams for relational MDPs. J. Artif. Intell. Res. (JAIR) 31, 431–472 (2008)
Boutilier, C., Reiter, R., Soutchanski, M., Thrun, S.: Decision-theoretic, high-level agent programming in the situation calculus. In: Proceedings of the Seventeenth National Conference on Artificial Intelligence (AAAI 2000) and of the Twelfth Conference on Innovative Applications of Artificial Intelligence (IAAI 2000), pp. 355–362. AAAI Press, Menlo Park (2000)
Littman, M., Majercik, S., Pitassi, T.: Stochastic boolean satisfiability. J. Autom. Reasoning 27, 251–296 (2001)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Rens, G., Meyer, T., Lakemeyer, G. (2015). A Logic for Reasoning About Decision-Theoretic Projections. In: Duval, B., van den Herik, J., Loiseau, S., Filipe, J. (eds) Agents and Artificial Intelligence. ICAART 2015. Lecture Notes in Computer Science(), vol 9494. Springer, Cham. https://doi.org/10.1007/978-3-319-27947-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-27947-3_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-27946-6
Online ISBN: 978-3-319-27947-3
eBook Packages: Computer ScienceComputer Science (R0)