Probabilistic Bisimulation: Naturally on Distributions

  • Holger Hermanns
  • Jan Krčál
  • Jan Křetínský
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8704)


In contrast to the usual understanding of probabilistic systems as stochastic processes, recently these systems have also been regarded as transformers of probabilities. In this paper, we give a natural definition of strong bisimulation for probabilistic systems corresponding to this view that treats probability distributions as first-class citizens. Our definition applies in the same way to discrete systems as well as to systems with uncountable state and action spaces. Several examples demonstrate that our definition refines the understanding of behavioural equivalences of probabilistic systems. In particular, it solves a longstanding open problem concerning the representation of memoryless continuous time by memoryfull continuous time. Finally, we give algorithms for computing this bisimulation not only for finite but also for classes of uncountably infinite systems.


Markov Decision Process Continuous Time Markov Chain Statistical Model Check Time Automaton Probabilistic Automaton 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agrawal, M., Akshay, S., Genest, B., Thiagarajan, P.: Approximate verification of the symbolic dynamics of Markov chains. In: LICS (2012)Google Scholar
  2. 2.
    Alur, R., Dill, D.: A theory of timed automata. Theor. Comput. Sci. 126(2), 183–235 (1994)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Behrmann, G., David, A., Larsen, K.G., Pettersson, P., Yi, W.: Developing UPPAAL over 15 years. Softw., Pract. Exper. 41(2), 133–142 (2011)CrossRefGoogle Scholar
  4. 4.
    Bernardo, M., Nicola, R.D., Loreti, M.: Revisiting bisimilarity and its modal logic for nondeterministic and probabilistic processes. Technical Report 06, IMT Lucca (2013)Google Scholar
  5. 5.
    Bravetti, M., D’Argenio, P.R.: Tutte le algebre insieme: Concepts, discussions and relations of stochastic process algebras with general distributions. In: Baier, C., Haverkort, B.R., Hermanns, H., Katoen, J.-P., Siegle, M. (eds.) Validation of Stochastic Systems. LNCS, vol. 2925, pp. 44–88. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Bravetti, M., Hermanns, H., Katoen, J.-P.: YMCA: Why Markov Chain Algebra? Electr. Notes Theor. Comput. Sci. 162, 107–112 (2006)CrossRefGoogle Scholar
  7. 7.
    Castro, P., Panangaden, P., Precup, D.: Equivalence relations in fully and partially observable Markov decision processes. In: IJCAI (2009)Google Scholar
  8. 8.
    Cattani, S.: Trace-based Process Algebras for Real-Time Probabilistic Systems. PhD thesis, University of Birmingham (2005)Google Scholar
  9. 9.
    Crafa, S., Ranzato, F.: A spectrum of behavioral relations over lTSs on probability distributions. In: Katoen, J.-P., König, B. (eds.) CONCUR 2011. LNCS, vol. 6901, pp. 124–139. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  10. 10.
    D’Argenio, P., Katoen, J.-P.: A theory of stochastic systems, part I: Stochastic automata. Inf. Comput. 203(1), 1–38 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    D’Argenio, P., Katoen, J.-P.: A theory of stochastic systems, part II: Process algebra. Inf. Comput. 203(1), 39–74 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    D’Argenio, P.R., Baier, C.: What is the relation between CTMC and TA? Personal Communication (1999)Google Scholar
  13. 13.
    David, A., Larsen, K.G., Legay, A., Mikučionis, M., Poulsen, D.B., van Vliet, J., Wang, Z.: Statistical model checking for networks of priced timed automata. In: Fahrenberg, U., Tripakis, S. (eds.) FORMATS 2011. LNCS, vol. 6919, pp. 80–96. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  14. 14.
    David, A., Larsen, K.G., Legay, A., Mikučionis, M., Wang, Z.: Time for statistical model checking of real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 349–355. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  15. 15.
    de Vink, E.P., Rutten, J.J.M.M.: Bisimulation for probabilistic transition systems: A coalgebraic approach. In: Degano, P., Gorrieri, R., Marchetti-Spaccamela, A. (eds.) ICALP 1997. LNCS, vol. 1256, pp. 460–470. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  16. 16.
    Deng, Y., Hennessy, M.: On the semantics of Markov automata. Inf. Comput. 222, 139–168 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Desharnais, J., Gupta, V., Jagadeesan, R., Panangaden, P.: Approximating labeled Markov processes. In: LICS (2000)Google Scholar
  18. 18.
    Doberkat, E.-E.: Semi-pullbacks and bisimulations in categories of stochastic relations. In: Baeten, J.C.M., Lenstra, J.K., Parrow, J., Woeginger, G.J. (eds.) ICALP 2003. LNCS, vol. 2719, pp. 996–1007. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  19. 19.
    Doyen, L., Henzinger, T., Raskin, J.-F.: Equivalence of labeled Markov chains. Int. J. Found. Comput. Sci. 19(3), 549–563 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  20. 20.
    Doyen, L., Massart, T., Shirmohammadi, M.: Limit synchronization in Markov decision processes. CoRR, abs/1310.2935 (2013)Google Scholar
  21. 21.
    Edalat, A.: Semi-pullbacks and bisimulation in categories of Markov processes. Mathematical Structures in Computer Science 9(5), 523–543 (1999)CrossRefzbMATHMathSciNetGoogle Scholar
  22. 22.
    Eisentraut, C., Hermanns, H., Krämer, J., Turrini, A., Zhang, L.: Deciding bisimilarities on distributions. In: QEST (2013)Google Scholar
  23. 23.
    Eisentraut, C., Hermanns, H., Zhang, L.: On probabilistic automata in continuous time. In: LICS (2010)Google Scholar
  24. 24.
    Feng, Y., Zhang, L.: When equivalence and bisimulation join forces in probabilistic automata. In: Jones, C., Pihlajasaari, P., Sun, J. (eds.) FM 2014. LNCS, vol. 8442, pp. 247–262. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  25. 25.
    Gast, N., Gaujal, B.: A mean field approach for optimization in discrete time. Discrete Event Dynamic Systems 21(1), 63–101 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  26. 26.
    Georgievska, S., Andova, S.: Probabilistic may/must testing: Retaining probabilities by restricted schedulers. Formal Asp. Comput. 24(4-6), 727–748 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  27. 27.
    Giry, M.: A categorical approach to probability theory. In: Banaschewski, B. (ed.) Categorical Aspects of Topology and Analysis. LNCS, vol. 915, pp. 68–85. Springer, Heidelberg (1982)Google Scholar
  28. 28.
    Harrison, P.G., Strulo, B.: Spades - a process algebra for discrete event simulation. J. Log. Comput. 10(1), 3–42 (2000)CrossRefzbMATHMathSciNetGoogle Scholar
  29. 29.
    Hennessy, M.: Exploring probabilistic bisimulations, part I. Formal Asp. Comput. (2012)Google Scholar
  30. 30.
    Hermanns, H., Herzog, U., Mertsiotakis, V.: Stochastic process algebras - between LOTOS and Markov chains. Computer Networks 30(9-10), 901–924 (1998)CrossRefGoogle Scholar
  31. 31.
    Hermanns, H., Krčál, J., Křetínský, J.: Probabilistic bisimulation: Naturally on distributions. CoRR, abs/1404.5084 (2014)Google Scholar
  32. 32.
    Hermanns, H., Turrini, A.: Deciding probabilistic automata weak bisimulation in polynomial time. In: FSTTCS (2012)Google Scholar
  33. 33.
    Hillston, J.: A Compositional Approach to Performance Modelling. Cambridge University Press, New York (1996)CrossRefGoogle Scholar
  34. 34.
    Jansen, D.N., Nielson, F., Zhang, L.: Belief bisimulation for hidden Markov models - logical characterisation and decision algorithm. In: Goodloe, A.E., Person, S. (eds.) NFM 2012. LNCS, vol. 7226, pp. 326–340. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  35. 35.
    Kerstan, H., König, B.: Coalgebraic trace semantics for probabilistic transition systems based on measure theory. In: Koutny, M., Ulidowski, I. (eds.) CONCUR 2012. LNCS, vol. 7454, pp. 410–424. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  36. 36.
    Korthikanti, V., Viswanathan, M., Agha, G., Kwon, Y.: Reasoning about MDPs as transformers of probability distributions. In: QEST (2010)Google Scholar
  37. 37.
    Kwiatkowska, M., Norman, G., Parker, D.: Prism 4.0: Verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  38. 38.
    Larsen, K., Skou, A.: Bisimulation through probabilistic testing. In: POPL (1989)Google Scholar
  39. 39.
    May, R., et al.: Biological populations with nonoverlapping generations: stable points, stable cycles, and chaos. Science 186(4164), 645–647 (1974)CrossRefGoogle Scholar
  40. 40.
    Milner, R.: Communication and concurrency. PHI Series in computer science. Prentice Hall (1989)Google Scholar
  41. 41.
    Mio, M.: Upper-expectation bisimilarity and Lukasiewicz μ-calculus. In: Muscholl, A. (ed.) FOSSACS 2014. LNCS, vol. 8412, pp. 335–350. Springer, Heidelberg (2014)Google Scholar
  42. 42.
    Sangiorgi, D., Rutten, J.: Advanced Topics in Bisimulation and Coinduction, 1st edn. Cambridge University Press, New York (2011)CrossRefGoogle Scholar
  43. 43.
    Segala, R.: Modeling and Verification of Randomized Distributed Real-time Systems. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA, USA (1995)Google Scholar
  44. 44.
    Segala, R., Lynch, N.: Probabilistic simulations for probabilistic processes. In: Jonsson, B., Parrow, J. (eds.) CONCUR 1994. LNCS, vol. 836, pp. 481–496. Springer, Heidelberg (1994)CrossRefGoogle Scholar
  45. 45.
    Shani, G., Pineau, J., Kaplow, R.: A survey of point-based pomdp solvers. AAMAS 27(1), 1–51 (2013)Google Scholar
  46. 46.
    Sokolova, A.: Probabilistic systems coalgebraically: A survey. Theor. Comput. Sci. 412(38), 5095–5110 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  47. 47.
    Song, L., Zhang, L., Godskesen, J.C.: Bisimulations meet PCTL equivalences for probabilistic automata. In: Katoen, J.-P., König, B. (eds.) CONCUR 2011. LNCS, vol. 6901, pp. 108–123. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  48. 48.
    Song, L., Zhang, L., Godskesen, J.C.: Late weak bisimulation for Markov automata. CoRR, abs/1202.4116 (2012)Google Scholar
  49. 49.
    Stoelinga, M., Vaandrager, F.: A testing scenario for probabilistic automata. In: Baeten, J.C.M., Lenstra, J.K., Parrow, J., Woeginger, G.J. (eds.) ICALP 2003. LNCS, vol. 2719, pp. 464–477. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  50. 50.
    Tzeng, W.: A polynomial-time algorithm for the equivalence of probabilistic automata. SIAM J. Comput. 21(2), 216–227 (1992)CrossRefzbMATHMathSciNetGoogle Scholar
  51. 51.
    Wolovick, N.: Continuous probability and nondeterminism in labeled transaction systems. PhD thesis, Universidad Nacional de Córdoba (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Holger Hermanns
    • 1
  • Jan Krčál
    • 1
  • Jan Křetínský
    • 2
  1. 1.Saarland University – Computer ScienceSaarbrückenGermany
  2. 2.ISTAustria

Personalised recommendations