Skip to main content
SpringerLink
Log in

Knowledge Representation for Stochastic Decision Processes

  • Chapter
  • First Online:
Artificial Intelligence Today

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 1600))

Abstract

Reasoning about stochastic dynamical systems and planning under uncertainty has come to play a fundamental role in AI research and applications. The representation of such systems, in particular, of actions with stochastic effects, has accordingly been given increasing attention in recent years. In this article, we survey a number of techniques for representing stochastic processes and actions with stochastic effects using dynamic Bayesian networks and influence diagrams, and briefly describe how these support effective inference for tasks such as monitoring, forecasting, explanation and decision making. We also compare these techniques to several action representations adopted in the classical reasoning about action and planning communities, describing how traditional problems such as the frame and ramification problems are dealt with in stochastic settings, and how these solutions compare to recent approaches to this problem in the classical (deterministic) literature. We argue that while stochastic dynamics introduce certain complications when it comes to such issues, for the most part, intuitions underlying classical models can be extended to the stochastic setting.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. James Allen, James Hendler, and Austin Tate, editors. Readings in Planning. Morgan-Kaufmann, San Mateo, 1990.

    Google Scholar 

  2. K. J. Astrom. Optimal control of Markov decision processes with incomplete state estimation. J. Math. Anal. Appl., 10:174–205, 1965.

    Article  MATH  Google Scholar 

  3. Andrew B. Baker. Nonmonotonic reasoning in the framework of the situation calculus. Artificial Intelligence, 49:5–23, 1991.

    Article  MathSciNet  MATH  Google Scholar 

  4. Richard E. Bellman. Dynamic Programming. Princeton University Press, Princeton, 1957.

    MATH  Google Scholar 

  5. Craig Boutilier, Thomas Dean, and Steve Hanks. Decision theoretic planning: Structural assumptions and computational leverage. Journal of Artificial Intelligence Research, 1998. To appear.

    Google Scholar 

  6. Craig Boutilier and Richard Dearden. Approximating value trees in structured dynamic programming. In Proceedings of the Thirteenth International Conference on Machine Learning, pages 54–62, Bari, Italy, 1996.

    Google Scholar 

  7. Craig Boutilier, Richard Dearden, and Moisés Goldszmidt. Exploiting structure in policy construction. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, pages 1104–1111, Montreal, 1995.

    Google Scholar 

  8. Craig Boutilier and Nir Friedman. Nondeterministic actions and the frame problem. In AAAI Spring Symposium on Extending Theories of Action: Formal Theory and Practical Applications, pages 39–44, Stanford, 1995.

    Google Scholar 

  9. Craig Boutilier, Nir Friedman, Moisés Goldszmidt, and Daphne Koller. Context-specific independence in Bayesian networks. In Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence, pages 115–123, Portland, OR, 1996.

    Google Scholar 

  10. Craig Boutilier and Moisés Goldszmidt. The frame problem and Bayesian network action representations. In Proceedings of the Eleventh Biennial Canadian Conference on Artificial Intelligence, pages 69–83, Toronto, 1996.

    Google Scholar 

  11. Craig Boutilier and David Poole. Computing optimal policies for partially observable decision processes using compact representations. In Proceedings of the Thirteenth National Conference on Artificial Intelligence, pages 1168–1175, Portland, OR, 1996.

    Google Scholar 

  12. Craig Boutilier and Martin L. Puterman. Process-oriented planning and average-reward optimality. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, pages 1096–1103, Montreal, 1995.

    Google Scholar 

  13. Xavier Boyen and Daphne Koller. Tractable inference for complex stochastic processes. In Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pages 33–42, Madison, WI, 1998.

    Google Scholar 

  14. Peter E. Caines. Linear stochastic systems. Wiley, New York, 1988.

    MATH  Google Scholar 

  15. Adrian Y. W. Cheuk and Craig Boutilier. Structured arcreversal and simulation of dynamic probabilistic networks. In Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence, pages 72–79, Providence, RI, 1997.

    Google Scholar 

  16. Thomas Dean and Keiji Kanazawa. A model for reasoning about persistence and causation. Computational Intelligence, 5(3):142–150, 1989.

    Article  Google Scholar 

  17. Thomas Dean and Michael Wellman. Planning and Control. Morgan Kaufmann, San Mateo, 1991.

    Google Scholar 

  18. Denise Draper, Steve Hanks, and Daniel Weld. A probabilistic model of action for least-commitment planning with information gathering. In Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence, pages 178–186, Seattle, 1994.

    Google Scholar 

  19. Steve Hanks (ed.). Decision theoretic planning: Proceedings of the aaai spring symposium. Technical Report SS-94-06, AAAI Press, Menlo Park, 1994.

    Google Scholar 

  20. Richard E. Fikes and Nils J. Nilsson. STRIPS: A new approach to the application of theorem proving to problem solving. Artificial Intelligence, 2:189–208, 1971.

    Article  MATH  Google Scholar 

  21. Dan Geiger and David Heckerman. Advances in probabilistic reasoning. In Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, pages 118–126, Los Angeles, 1991.

    Google Scholar 

  22. Michael Gelfond and Vladimir Lifschitz. Representing actions in extended logic programming. In K. Apt, editor, Proceedings of the Tenth Conference on Logic Programming, pages 559–573, 1992.

    Google Scholar 

  23. Enrico Giunchiglia, G. Neelakantan Kartha, and Vladimir Lifschitz. Representing action: Indeterminacy and ramifications. Artificial Intelligence, 95:409–438, 1997.

    Article  MathSciNet  MATH  Google Scholar 

  24. Steve Hanks and Drew V. McDermott. Modeling a dynamic and uncertain world i: Symbolic and probabilistic reasoning about change. Artificial Intelligence, 1994.

    Google Scholar 

  25. David Heckerman. Probabilistic Similarity Networks. PhD thesis, Stanford University, Stanford, 1990.

    Google Scholar 

  26. Ronald A. Howard. Dynamic Programming and Markov Processes. MIT Press, Cambridge, 1960.

    MATH  Google Scholar 

  27. Ronald A. Howard and James E. Matheson, editors. Readings on the Principles and Applications of Decision Analysis. Strategic Decision Group, Menlo Park, CA, 1984.

    Google Scholar 

  28. Manfred Jaeger. Relational Bayesian networks. In Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence, pages 266–273, Providence, RI, 1997.

    Google Scholar 

  29. R. E. Kalman. A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 82:35–45, 1960.

    Article  Google Scholar 

  30. Keiji Kanazawa, Daphne Koller, and Stuart Russell. Stochastic simulation algorithms for dynamic probabilistic networks. In Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, pages 346–351, Montreal, 1995.

    Google Scholar 

  31. G. Neelakantan Kartha. Two counterexamples related to Baker’s approach to the frame problem. Artificial Intelligence, 69:379–392, 1994.

    Article  MathSciNet  MATH  Google Scholar 

  32. G. Neelakantan Kartha and Vladimir Lifschitz. Actions with indirect effects (preliminary report). In Proceedings of the Fifth International Conference on Principles of Knowledge Representation and Reasoning, pages 341–350, Bonn, 1994.

    Google Scholar 

  33. Henry A. Kautz. The logic of persistence. In Proceedings of the Fifth National Conference on Artificial Intelligence, pages 401–405, Philadelphia, 1986.

    Google Scholar 

  34. John G. Kemeny and J. Laurie Snell. Finite Markov Chains. Van Nostrand, Princeton, NJ, 1960.

    MATH  Google Scholar 

  35. Uffe Kjaerulff. A computational scheme for reasoning in dynamic probabilistic networks. In Proceedings of the Eighth Conference on Uncertainty in AI, pages 121–129, Stanford, 1992.

    Google Scholar 

  36. Nicholas Kushmerick, Steve Hanks, and Daniel Weld. An algorithm for probabilistic least-commitment planning. In Proceedings of the Twelfth National Conference on Artificial Intelligence, pages 1073–1078, Seattle, 1994.

    Google Scholar 

  37. Nicholas Kushmerick, Steve Hanks, and Daniel Weld. An algorithm for probabilistic planning. Artificial Intelligence, 76:239–286, 1995.

    Article  Google Scholar 

  38. S. L. Lauritzen. Propagation of probabilities, means and variances in mixed graphical association models. Journal of the American Statistical Association, 87:1098–1108, 1992.

    Article  MathSciNet  MATH  Google Scholar 

  39. Fangzhen Lin and Ray Reiter. State constraints revisited. Journal of Logic and Computation, 4(5):655–678, 1994.

    Article  MathSciNet  MATH  Google Scholar 

  40. D. G. Luenberger. Introduction to Dynamic Systems: Theory, Models and Applications. Wiley, New York, 1979.

    MATH  Google Scholar 

  41. John McCarthy and P.J. Hayes. Some philosophical problems from the standpoint of artificial intelligence. Machine Intelligence, 4:463–502, 1969.

    MATH  Google Scholar 

  42. A. E. Nicholson and J. M. Brady. Sensor validation using dynamic belief networks. In Proceedings of the Eighth Conference on Uncertainty in AI, pages 207–214, Stanford, 1992.

    Google Scholar 

  43. Judea Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Mateo, 1988.

    MATH  Google Scholar 

  44. Edwin Pednault. ADL: Exploring the middle ground between STRIPS and the situation calculus. In Proceedings of the First International Conference on Principles of Knowledge Representation and Reasoning, pages 324–332, Toronto, 1989.

    Google Scholar 

  45. Mark A. Peot and David E. Smith. Conditional nonlinear planning. In Proceedings of the First International Conference on AI Planning Systems, pages 189–197, College Park, MD, 1992.

    Google Scholar 

  46. David Poole. Probabilistic Horn abduction and Bayesian networks. Artificial Intelligence, 64(1):81–129, 1993.

    Article  MATH  Google Scholar 

  47. David Poole. Exploiting the rule structure for decision making within the independent choice logic. In Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, pages 454–463, Montreal, 1995.

    Google Scholar 

  48. David Poole. The independent choice logic for modelling multiple agents under uncertainty. Artificial Intelligence, 94(l–2):7–56, 1997.

    Article  MathSciNet  MATH  Google Scholar 

  49. David Poole. Probabilistic partial evaluation: Exploiting rule structure in probabilistic inference. In Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, pages 1284–1291, Nagoya, 1997.

    Google Scholar 

  50. Martin L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York, 1994.

    Book  MATH  Google Scholar 

  51. Lawrence R. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257–286, 1989.

    Article  Google Scholar 

  52. Raymond Reiter. The frame problem in the situation calculus: A simple solution (sometimes) and a completeness result for goal regression. In V. Lifschitz, editor, Artificial Intelligence and Mathematical Theory of Computation (Papers in Honor of John McCarthy), pages 359–380. Academic Press, San Diego, 1991.

    Chapter  Google Scholar 

  53. Erik Sandewall. Features and Fluents. Oxford University Press, Oxford, 1995.

    MATH  Google Scholar 

  54. Lenhart K. Schubert. Monotonic solution of the frame problem in the situation calculus: An efficient method for worlds with fully specified actions. In H. E. Kyburg, R. P. Loui, and G. N. Carlson, editors, Knowledge Representation and Defeasible Reasoning, pages 23–67. Kluwer, Boston, 1990.

    Chapter  Google Scholar 

  55. R. D. Shachter and C. R. Kenley. Gaussian influence diagrams. Management Science, 35(5):527–550, 1989.

    Article  Google Scholar 

  56. Ross D. Shachter. Evaluating influence diagrams. Operations Research, 33(6):871–882, 1986.

    Article  MathSciNet  Google Scholar 

  57. Solomon E. Shimony. The role of relevance in explanation I: Irrelevance as statistical independence. International Journal of Approximate Reasoning, 8(4):281–324, 1993.

    Article  MathSciNet  MATH  Google Scholar 

  58. Yoav Shoham. Reasoning About Change: Time and Causation from the Standpoint of Artificial Intelligence. MIT Press, Cambridge, 1988.

    Google Scholar 

  59. Richard D. Smallwood and Edward J. Sondik. The optimal control of partially observable Markov processes over a finite horizon. Operations Research, 21:1071–1088, 1973.

    Article  MATH  Google Scholar 

  60. David E. Smith and Daniel S. Weld. Conformant graphplan. In Proceedings of the Fifteenth National Conference on Artificial Intelligence, pages 889–896, Madison, 1998.

    Google Scholar 

  61. Joseph A. Tatman and Ross D. Shachter. Dynamic programming and influence diagrams. IEEE Transactions on Systems, Man and Cybernetics, 20(2):365–379, 1990.

    Article  MathSciNet  MATH  Google Scholar 

  62. Michael Thielscher. Ramification and causality. Artificial Intelligence, 89:317–364, 1997.

    Article  MathSciNet  MATH  Google Scholar 

  63. D. Warren. Generating Conditional Plans and Programs. In Proceedings of AISB Summer Conference, pages 344–354, University of Edinburgh, 1976.

    Google Scholar 

  64. M. West and J. Harrison. Bayesian Forecasting and Dynamic Models. Springer-Verlag, New York, 1989.

    Book  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Computer Science, University of British Columbia, Vancouver, BC, V6T 1Z4, Canada

    Craig Boutilier

Authors
  1. Craig Boutilier
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Electronic Engineering, Queen Mary and Westfield College, University of London, London, E1 4NS, UK

    Michael J. Wooldridge

  2. Department of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    Manuela Veloso

Rights and permissions

Reprints and permissions

Copyright information

© 1999 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Boutilier, C. (1999). Knowledge Representation for Stochastic Decision Processes. In: Wooldridge, M.J., Veloso, M. (eds) Artificial Intelligence Today. Lecture Notes in Computer Science(), vol 1600. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48317-9_5

Download citation

  • DOI: https://doi.org/10.1007/3-540-48317-9_5

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-66428-4

  • Online ISBN: 978-3-540-48317-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation