Markov Decision Processes from Colored Petri Nets

  • Monica Góes Eboli
  • Fabio Gagliardi Cozman
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6404)


Models that are suitable for planning are not always easy to specify. In this paper we investigate the conversion of Petri nets into factored Markov decision processes: the former are relatively easy to build while the latter are adequate for policy generation. To represent probabilities that are needed when planning under uncertainty, we introduce factored Petri nets; we then describe the conversion of factored Petri nets in Markov decision processes.


Planning under uncertainty Markov decision processes Colored Petri nets 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Marsan, M.A., Balbo, G., Conte, G., Donatelli, S., Franceschinis, G.: Modeling with Generalized Stochastic Petri Nets. John Wiley and Sons, Chichester (1995)zbMATHGoogle Scholar
  2. 2.
    Zhou, M., Venkatesh, K.: Modeling, Simulation and Control of Flexible Manufacturing Systems – A Petri Net Approach. World Scientific, Singapore (1999)CrossRefGoogle Scholar
  3. 3.
    Hass, P.J.: Stochastic Petri nets for modeling and simulation. In: Winter Simulation Conference, Washington, pp. 101–112 (2004)Google Scholar
  4. 4.
    Jensen, K.: An introduction to the theoretical aspects of colored Petri Nets. In: de Bakker, J.W., de Roever, W.-P., Rozenberg, G. (eds.) REX 1993. LNCS, vol. 803, pp. 230–272. Springer, Heidelberg (1994)CrossRefGoogle Scholar
  5. 5.
    Jensen, K.: An introduction to the practical use of colored Petri nets, pp. 237–292. Springer, Berlin (1998)Google Scholar
  6. 6.
    Jensen, K., Kristensen, L.M., Cristensen, S.: The Practioners Guide to Coloured Petri Nets, pp. 98–132. Springer, Berlin (1998)Google Scholar
  7. 7.
    Miyagi, P.E.: Controle Programavel. Editora Edgard Blcher (1996)Google Scholar
  8. 8.
    Russel, S., Norvig, P.: Inteligencia Artificial. Editora Campus (2004)Google Scholar
  9. 9.
    Puterman, M.L.: Markov Decision Processes. Wiley Series in Probability and Mathematical Statistics. John Wiley and Sons, New York (1994)CrossRefzbMATHGoogle Scholar
  10. 10.
    Trevisan, F.W., Cozman, F.G., Barros, L.N.: Planning under risk and Knightian uncertainty. In: International Joint Conference on Artificial Intelligence, pp. 2023–2028 (2007)Google Scholar
  11. 11.
    Delgado, K.V., Barros, L.N., Cozman, F.G., Shirota, R.: Representing and solving factored Markov decision processes with imprecise probabilities. In: 6th International Symposium on Imprecise Probability: Theories and Applications, Durham, United Kingdom, pp. 169–178 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Monica Góes Eboli
    • 1
  • Fabio Gagliardi Cozman
    • 1
  1. 1.Escola Politécnica da Universidade de São PauloBrazil

Personalised recommendations