Using Conditional Random Fields for Decision-Theoretic Planning

  • Paul A. Ardis
  • Christopher M. Brown
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5861)


We propose a means of extending Conditional Random Field modeling to decision-theoretic planning where valuation is dependent upon fully-observable factors. Representation is discussed, and a comparison with existing decision problem methodologies is presented. Included are exact and inexact message passing schemes for policy making, examples of decision making in practice, extensions to solving general decision problems, and suggestions for future use.


Utility and Decision Theory Graphical Modeling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Demirer, R., Shenoy, P.P.: Sequential valuations networks for asymmetric decision problems. European Journal of Operational Research 169(1), 286–309 (2006)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Jensen, F.V., Vomlelova, M.: Unconstrained influence diagrams. In: Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence, pp. 234–241 (2002)Google Scholar
  3. 3.
    Kearns, M., Mansour, Y., Ng, A.Y.: A sparse sampling algorithm for near-optimal planning in large markov decision processes. In: Proceedings of the International Joint Conference on Artificial Intelligence 1999, pp. 1324–1331 (1999)Google Scholar
  4. 4.
    Kindermann, R., Snell, J.L.: Markov Random Fields and their Applications. American Mathematical Society, Providence (1980)Google Scholar
  5. 5.
    Kschischang, F.R., Frey, B.J., Loeliger, H.A.: Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory 47(2), 498–519 (2001)zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Lafferty, J., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the 18th International Conference on Machine Learning, pp. 282–289 (2001)Google Scholar
  7. 7.
    Meuleau, N., Hauskrecht, M., Kim, K.-E., Peshkin, L., Kaelbling, L.P., Dean, T., Boutilier, C.: Solving very large weakly coupled markov decision processes. In: Proceedings of the Fifteenth National Conference on Artificial Intelligence, pp. 165–172 (1998)Google Scholar
  8. 8.
    Pal, C., Sutton, C., McCallum, A.: Sparse forward-backward using minimum divergence beams for fast training of conditional random fields. In: Proceedings of the 2006 International Conference on Acoustics, Speech, and Signal Processing, vol. 5, pp. 581–584 (2006)Google Scholar
  9. 9.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)Google Scholar
  10. 10.
    Richardson, M., Domingos, P.: Markov logic networks. Machine Learning 62, 107–136 (2006)CrossRefGoogle Scholar
  11. 11.
    Shenoy, P.P.: Valuation network representation and solution of asymmetric decision problems. European Journal of Operational Research 121(3), 579–608 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Sutton, C., McCallum, A., Rohanimanesh, K.: Dynamic conditional random fields: Factorized probabilistic models for labeling and segmenting sequential data. The Journal of Machine Learning Research 8, 693–723 (2007)Google Scholar
  13. 13.
    Weinman, J., Tran, L., Pal, C.: Efficiently learning random fields for stereo vision with sparse message passing. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 617–630. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  14. 14.
    Yedidia, J.S., Freeman, W.T., Weiss, Y.: Generalized belief propagation. In: Neural Information Processing Systems, pp. 689–695 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Paul A. Ardis
    • 1
  • Christopher M. Brown
    • 1
  1. 1.University of RochesterRochester

Personalised recommendations