Counterexample Generation for Discrete-Time Markov Models: An Introductory Survey

  • Erika Ábrahám
  • Bernd Becker
  • Christian Dehnert
  • Nils Jansen
  • Joost-Pieter Katoen
  • Ralf Wimmer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8483)


This paper is an introductory survey of available methods for the computation and representation of probabilistic counterexamples for discrete-time Markov chains and probabilistic automata. In contrast to traditional model checking, probabilistic counterexamples are sets of finite paths with a critical probability mass. Such counterexamples are not obtained as a by-product of model checking, but by dedicated algorithms. We define what probabilistic counterexamples are and present approaches how they can be generated. We discuss methods based on path enumeration, the computation of critical subsystems, and the generation of critical command sets, both, using explicit and symbolic techniques.


Model Check Target State Mixed Integer Linear Programming Probabilistic Program Probable Path 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Clarke, E.M.: The birth of model checking. In: Grumberg, O., Veith, H. (eds.) 25 Years of Model Checking. LNCS, vol. 5000, pp. 1–26. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  2. 2.
    Clarke, E.M., Veith, H.: Counterexamples revisited: Principles, algorithms, applications. In: Dershowitz, N. (ed.) Verification: Theory and Practice. LNCS, vol. 2772, pp. 208–224. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  3. 3.
    Fraser, G., Wotawa, F., Ammann, P.: Issues in using model checkers for test case generation. Journal of Systems and Software 82(9), 1403–1418 (2009)CrossRefGoogle Scholar
  4. 4.
    Behrmann, G., Larsen, K.G., Rasmussen, J.I.: Optimal scheduling using priced timed automata. SIGMETRICS Performance Evaluation Review 32(4), 34–40 (2005)CrossRefGoogle Scholar
  5. 5.
    Ngo, T.M., Stoelinga, M., Huisman, M.: Effective verification of confidentiality for multi-threaded programs. Journal of Computer Security 22(2), 269–300 (2014)CrossRefGoogle Scholar
  6. 6.
    Clarke, E.M., Grumberg, O., Jha, S., Lu, Y., Veith, H.: Counterexample-guided abstraction refinement for symbolic model checking. J. ACM 50(5), 752–794 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Clarke, E.M., Jha, S., Lu, Y., Veith, H.: Tree-like counterexamples in model checking. In: Proc. of LICS, pp. 19–29. IEEE Computer Society Press (2002)Google Scholar
  8. 8.
    Clarke, E.M., Grumberg, O., McMillan, K.L., Zhao, X.: Efficient generation of counterexamples and witnesses in symbolic model checking. In: Proc. of DAC, pp. 427–432 (1995)Google Scholar
  9. 9.
    Baier, C., Katoen, J.P.: Principles of Model Checking. The MIT Press (2008)Google Scholar
  10. 10.
    Baier, C., Haverkort, B.R., Hermanns, H., Katoen, J.P.: Performance evaluation and model checking join forces. Commun. ACM 53(9), 76–85 (2010)CrossRefGoogle Scholar
  11. 11.
    Kwiatkowska, M.Z.: Model checking for probability and time: From theory to practice. In: Proc. of LICS, pp. 351–360. IEEE Computer Society Press (2003)Google Scholar
  12. 12.
    Kwiatkowska, M.Z., Norman, G., Parker, D.: PRISM 4.0: Verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  13. 13.
    Katoen, J.P., Zapreev, I.S., Hahn, E.M., Hermanns, H., Jansen, D.N.: The ins and outs of the probabilistic model checker MRMC. Perform. Eval. 68(2), 90–104 (2011)CrossRefGoogle Scholar
  14. 14.
    Kwiatkowska, M.Z., Norman, G., Parker, D.: Stochastic model checking. In: Bernardo, M., Hillston, J. (eds.) SFM 2007. LNCS, vol. 4486, pp. 220–270. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  15. 15.
    Katoen, J.P.: Model checking meets probability: A gentle introduction. In: Engineering Dependable Software Systems. NATO Science for Peace and Security Series - D: Information and Communication Security, vol. 34, pp. 177–205. IOS Press, Amsterdam (2013)Google Scholar
  16. 16.
    Han, T., Katoen, J.P.: Counterexamples in probabilistic model checking. In: Grumberg, O., Huth, M. (eds.) TACAS 2007. LNCS, vol. 4424, pp. 72–86. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  17. 17.
    Han, T., Katoen, J.P., Damman, B.: Counterexample generation in probabilistic model checking. IEEE Transactions on Software Engineering 35(2), 241–257 (2009)CrossRefGoogle Scholar
  18. 18.
    Aljazzar, H., Leue, S.: Extended directed search for probabilistic timed reachability. In: Asarin, E., Bouyer, P. (eds.) FORMATS 2006. LNCS, vol. 4202, pp. 33–51. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  19. 19.
    Wimmer, R., Braitling, B., Becker, B.: Counterexample generation for discrete-time markov chains using bounded model checking. In: Jones, N.D., Müller-Olm, M. (eds.) VMCAI 2009. LNCS, vol. 5403, pp. 366–380. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  20. 20.
    Braitling, B., Wimmer, R., Becker, B., Jansen, N., Ábrahám, E.: Counterexample generation for markov chains using SMT-based bounded model checking. In: Bruni, R., Dingel, J. (eds.) FMOODS/FORTE 2011. LNCS, vol. 6722, pp. 75–89. Springer, Heidelberg (2011)Google Scholar
  21. 21.
    Günther, M., Schuster, J., Siegle, M.: Symbolic calculation of k-shortest paths and related measures with the stochastic process algebra tool Caspa. In: Proc. of DYADEM-FTS, pp. 13–18. ACM Press (2010)Google Scholar
  22. 22.
    Damman, B., Han, T., Katoen, J.P.: Regular expressions for PCTL counterexamples. In: Proc. of QEST, pp. 179–188. IEEE Computer Society Press (2008)Google Scholar
  23. 23.
    Daws, C.: Symbolic and parametric model checking of discrete-time Markov chains. In: Liu, Z., Araki, K. (eds.) ICTAC 2004. LNCS, vol. 3407, pp. 280–294. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  24. 24.
    Andrés, M.E., D’Argenio, P., van Rossum, P.: Significant diagnostic counterexamples in probabilistic model checking. In: Chockler, H., Hu, A.J. (eds.) HVC 2008. LNCS, vol. 5394, pp. 129–148. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  25. 25.
    Aljazzar, H., Leue, S.: Generation of counterexamples for model checking of Markov decision processes. In: Proc. of QEST, pp. 197–206. IEEE Computer Society Press (2009)Google Scholar
  26. 26.
    Chadha, R., Viswanathan, M.: A counterexample-guided abstraction-refinement framework for Markov decision processes. ACM Transactions on Computational Logic 12(1), 1–45 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Wimmer, R., Jansen, N., Ábrahám, E., Becker, B., Katoen, J.-P.: Minimal critical subsystems for discrete-time Markov models. In: Flanagan, C., König, B. (eds.) TACAS 2012. LNCS, vol. 7214, pp. 299–314. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  28. 28.
    Wimmer, R., Becker, B., Jansen, N., Ábrahám, E., Katoen, J.P.: Minimal critical subsystems as counterexamples for ω-regular DTMC properties. In: Proc. of MBMV, pp. 169–180. Verlag Dr. Kovač (2012)Google Scholar
  29. 29.
    Jansen, N., Ábrahám, E., Katelaan, J., Wimmer, R., Katoen, J.P., Becker, B.: Hierarchical counterexamples for discrete-time Markov chains. In: Bultan, T., Hsiung, P.-A. (eds.) ATVA 2011. LNCS, vol. 6996, pp. 443–452. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  30. 30.
    Aljazzar, H., Leue, S.: Directed explicit state-space search in the generation of counterexamples for stochastic model checking. IEEE Transactions on Software Engineering 36(1), 37–60 (2010)CrossRefGoogle Scholar
  31. 31.
    Jansen, N., Ábrahám, E., Zajzon, B., Wimmer, R., Schuster, J., Katoen, J.P., Becker, B.: Symbolic counterexample generation for discrete-time Markov chains. In: Păsăreanu, C.S., Salaün, G. (eds.) FACS 2012. LNCS, vol. 7684, pp. 134–151. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  32. 32.
    Jansen, N., Wimmer, R., Ábrahám, E., Zajzon, B., Katoen, J.P., Becker, B., Schuster, J.: Symbolic counterexample generation for large discrete-time Markov chains. In: Science of Computer Programming (2014) (accepted for publication)Google Scholar
  33. 33.
    Alur, R., Henzinger, T.A.: Reactive modules. Formal Methods in System Design 15(1), 7–48 (1999)CrossRefGoogle Scholar
  34. 34.
    Wimmer, R., Jansen, N., Vorpahl, A., Ábrahám, E., Katoen, J.-P., Becker, B.: High-level counterexamples for probabilistic automata. In: Joshi, K., Siegle, M., Stoelinga, M., D’Argenio, P.R. (eds.) QEST 2013. LNCS, vol. 8054, pp. 39–54. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  35. 35.
    Katoen, J.P., van de Pol, J., Stoelinga, M., Timmer, M.: A linear process-algebraic format with data for probabilistic automata. Theor. Comput. Sci. 413(1), 36–57 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Aljazzar, H., Leitner-Fischer, F., Leue, S., Simeonov, D.: DiPro - A tool for probabilistic counterexample generation. In: Groce, A., Musuvathi, M. (eds.) SPIN Workshops 2011. LNCS, vol. 6823, pp. 183–187. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  37. 37.
    Jansen, N., Ábrahám, E., Volk, M., Wimmer, R., Katoen, J.-P., Becker, B.: The COMICS tool – Computing minimal counterexamples for DTMCs. In: Chakraborty, S., Mukund, M. (eds.) ATVA 2012. LNCS, vol. 7561, pp. 349–353. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  38. 38.
    Hermanns, H., Wachter, B., Zhang, L.: Probabilistic CEGAR. In: Gupta, A., Malik, S. (eds.) CAV 2008. LNCS, vol. 5123, pp. 162–175. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  39. 39.
    Komuravelli, A., Păsăreanu, C.S., Clarke, E.M.: Assume-guarantee abstraction refinement for probabilistic systems. In: Madhusudan, P., Seshia, S.A. (eds.) CAV 2012. LNCS, vol. 7358, pp. 310–326. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  40. 40.
    Grunske, L., Winter, K., Yatapanage, N., Zafar, S., Lindsay, P.A.: Experience with fault injection experiments for FMEA. Softw. Pract. Exper. 41(11), 1233–1258 (2011)CrossRefGoogle Scholar
  41. 41.
    Aljazzar, H., Fischer, M., Grunske, L., Kuntz, M., Leitner-Fischer, F., Leue, S.: Safety analysis of an airbag system using probabilistic FMEA and probabilistic counterexamples. In: Proc. of QEST, pp. 299–308. IEEE Computer Society Press (2009)Google Scholar
  42. 42.
    Debbi, H., Bourahla, M.: Generating diagnoses for probabilistic model checking using causality. Journal of Computing and Information Technology 21(1), 13–23 (2013)CrossRefGoogle Scholar
  43. 43.
    Debbi, H., Bourahla, M.: Causal analysis of probabilistic counterexamples. In: Proc. of MEMOCODE, pp. 77–86. IEEE (2013)Google Scholar
  44. 44.
    Leitner-Fischer, F., Leue, S.: Probabilistic fault tree synthesis using causality computation. Int’l Journal of Critical Computer-Based Systems 4(2), 119–143 (2013)CrossRefGoogle Scholar
  45. 45.
    Bernardo, M., Hillston, J. (eds.): SFM 2007. LNCS, vol. 4486. Springer, Heidelberg (2007)Google Scholar
  46. 46.
    Kemeney, J.G., Snell, J.L., Knapp, A.W.: Denumerable Markov Chains. Springer (1976)Google Scholar
  47. 47.
    Segala, R., Lynch, N.A.: Probabilistic simulations for probabilistic processes. Nordic Journal on Computing 2(2), 250–273 (1995)MathSciNetzbMATHGoogle Scholar
  48. 48.
    Eppstein, D.: Finding the k shortest paths. SIAM Journal on Computing 28(2), 652–673 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  49. 49.
    Jiménez, V.M., Marzal, A.: Computing the k shortest paths: A new algorithm and an experimental comparison. In: Vitter, J.S., Zaroliagis, C.D. (eds.) WAE 1999. LNCS, vol. 1668, pp. 15–29. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  50. 50.
    Aljazzar, H., Leue, S.: K ∗ : A heuristic search algorithm for finding the k shortest paths. Artificial Intelligence 175(18), 2129–2154 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  51. 51.
    Biere, A., Cimatti, A., Clarke, E.M., Strichman, O., Zhu, Y.: Bounded model checking. Advances in Computers 58, 118–149 (2003)Google Scholar
  52. 52.
    Tseitin, G.S.: On the complexity of derivation in propositional calculus. Studies in Constructive Mathematics and Mathematical, Logic Part 2, 115–125 (1970)Google Scholar
  53. 53.
    Braitling, B., Wimmer, R., Becker, B., Ábrahám, E.: Stochastic bounded model checking: Bounded rewards and compositionality. In: Proc. of MBMV, pp. 243–254. Universität Rostock, ITMZ (2013)Google Scholar
  54. 54.
    Dijkstra, E.W.: A note on two problems in connexion with graphs. Numerische Mathematik 1, 269–271 (1959)MathSciNetCrossRefzbMATHGoogle Scholar
  55. 55.
    Daws, C.: Symbolic and parametric model checking of discrete-time Markov chains. In: Liu, Z., Araki, K. (eds.) ICTAC 2004. LNCS, vol. 3407, pp. 280–294. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  56. 56.
    Han, Y.S., Wood, D.: Obtaining shorter regular expressions from finite-state automata. Theoretical Computer Science 370(1-3), 110–120 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  57. 57.
    Halpern, J.Y., Pearl, J.: Causes and explanations: A structural approach. Part I: Causes. British Journal on the Philosophy of Science 56, 843–887 (2005)Google Scholar
  58. 58.
    Chockler, H., Halpern, J.Y.: Responsibility and blame: A structural-model approach. Journal of Artificial Intellelligence Research (JAIR) 22, 93–115 (2004)MathSciNetzbMATHGoogle Scholar
  59. 59.
    Leitner-Fischer, F., Leue, S.: On the synergy of probabilistic causality computation and causality checking. In: Bartocci, E., Ramakrishnan, C.R. (eds.) SPIN 2013. LNCS, vol. 7976, pp. 246–263. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  60. 60.
    Aljazzar, H., Leue, S.: Debugging of dependability models using interactive visualization of counterexamples. In: Proc. of QEST, pp. 189–198. IEEE Computer Society Press (2008)Google Scholar
  61. 61.
    Wimmer, R., Jansen, N., Ábrahám, E., Katoen, J.P., Becker, B.: Minimal counterexamples for refuting ω-regular properties of Markov decision processes (extended version). Reports of SFB/TR 14 AVACS 88 (2012) ISSN: 1860-9821,
  62. 62.
    Schrijver, A.: Theory of Linear and Integer Programming. Wiley (1986)Google Scholar
  63. 63.
    Pearl, J.: Heuristics: Intelligent Search Strategies for Computer Problem Solving. Addison-Wesley Longman Publishing Co. Inc., Boston (1984)Google Scholar
  64. 64.
    Baier, C., Clarke, E.M., Hartonas-Garmhausen, V., Kwiatkowska, M.Z., Ryan, M.: Symbolic model checking for probabilistic processes. In: Degano, P., Gorrieri, R., Marchetti-Spaccamela, A. (eds.) ICALP 1997. LNCS, vol. 1256, pp. 430–440. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  65. 65.
    Parker, D.: Implementation of Symbolic Model Checking for Probabilistic Systems. PhD thesis, University of Birmingham (2002)Google Scholar
  66. 66.
    Ábrahám, E., Jansen, N., Wimmer, R., Katoen, J.P., Becker, B.: DTMC model checking by SCC reduction. In: Proc. of QEST, pp. 37–46. IEEE Computer Society Press (2010)Google Scholar
  67. 67.
    Aljazzar, H., Leue, S.: K*: A directed on-the-fly algorithm for finding the k shortest paths. Technical report, Chair of Software Engineering, University of Konstanz, Germany (2008)Google Scholar
  68. 68.
    de Moura, L.M., Bjørner, N.: Z3: An efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  69. 69.
    Achterberg, T.: SCIP: Solving constraint integer programs. Mathematical Programming Computation 1(1), 1–41 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  70. 70.
  71. 71.
    Gurobi Optimization, Inc.: Gurobi optimizer reference manual (2013),

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Erika Ábrahám
    • 1
  • Bernd Becker
    • 2
  • Christian Dehnert
    • 1
  • Nils Jansen
    • 1
  • Joost-Pieter Katoen
    • 1
  • Ralf Wimmer
    • 2
  1. 1.RWTH Aachen UniversityGermany
  2. 2.Albert-Ludwigs-University FreiburgGermany

Personalised recommendations