Model Checking Markov Chains Using Krylov Subspace Methods: An Experience Report

  • Falko Dulat
  • Joost-Pieter Katoen
  • Viet Yen Nguyen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6342)


The predominant technique for computing the transient distribution of a Continuous Time Markov Chain (CTMC) exploits uniformization, which is known to be stable and efficient for non-stiff to mildly-stiff CTMCs. On stiff CTMCs however, uniformization suffers from severe performance degradation. In this paper, we report on our observations and analysis of an alternative technique using Krylov subspaces. We implemented a Krylov-based extension to MRMC (Markov Reward Model Checker) and conducted extensive experiments on five case studies from different application domains. The results reveal that the Krylov-based technique is an order of magnitude faster on stiff CTMCs.


Model Check Krylov Subspace Posteriori Error Estimate Continuous Time Markov Chain Krylov Subspace Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abdallah, H., Marie, R.: The uniformized power method for transient solutions of Markov processes. Computers & Operations Research 20(5), 515–526 (1993)CrossRefzbMATHGoogle Scholar
  2. 2.
    Baier, C., Haverkort, B., Hermanns, H., Katoen, J.-P.: Model-checking algorithms for continuous-time Markov chains. IEEE Trans. on Softw. Eng. 29(6), 524–541 (2003)CrossRefzbMATHGoogle Scholar
  3. 3.
    Bosnacki, D., Edelkamp, S., Sulewski, D.: Efficient probabilistic model checking on general purpose graphics processors. In: Păsăreanu, C.S. (ed.) SPIN Workshop. LNCS, vol. 5578, pp. 32–49. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  4. 4.
    Busch, H., Sandmann, W., Wolf, V.: A numerical aggregation algorithm for the enzyme-catalyzed substrate conversion. In: Priami, C. (ed.) CMSB 2006. LNCS (LNBI), vol. 4210, pp. 298–311. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  5. 5.
    Carrasco, J.A.: Transient analysis of rewarded continuous time Markov models by regenerative randomization with laplace transform inversion. The Conputer Journal 46(1), 84–99 (2003)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Chen, T., Han, T., Katoen, J.-P., Mereacre, A.: Quantitative model checking of continuous-time Markov chains against timed automata specification. In: LICS, pp. 309–318 (2009)Google Scholar
  7. 7.
    de Souza e Silva, E., Gail, H.R.: Transient solutions for Markov chains. In: Grassmann, W. (ed.) Computational Probability, pp. 43–81. Kluwer Academic Publishers, Dordrecht (2000)CrossRefGoogle Scholar
  8. 8.
    Duff, I., Grimes, R., Lewis, J.: User’s guide for the Harwell-Boeing sparse matrix collection. Technical Report TR/PA/92/86, CERFACS (1992)Google Scholar
  9. 9.
    Fox, B.L., Glynn, P.W.: Computing Poisson probabilities. Commun. ACM 31(4), 440–445 (1988)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Garren, S.T., Smith, R.L.: Estimating the second largest eigenvalue of a Markov transition matrix. Bernoulli 6(2), 215–242 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Henzinger, T.A., Mateescu, M., Wolf, V.: Sliding window abstraction for infinite Markov chains. In: Bouajjani, A., Maler, O. (eds.) Computer Aided Verification. LNCS, vol. 5643, pp. 337–352. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  12. 12.
    Hermanns, H., Meyer-Kayser, J., Siegle, M.: Multi terminal binary decision diagrams to represent and analyse continuous time Markov chains. In: Plateau, B., Stewart, W., Silva, M. (eds.) Proc. 3rd International Workshop on Numerical Solution of Markov Chains (NSMC 1999), pp. 188–207. Prensas Universitarias de Zaragoza (1999)Google Scholar
  13. 13.
    Hochbruck, M., Lubich, C.: On Krylov subspace approximations to the matrix exponential operator. SIAM on Numerical Analysis 34(5), 1911–1925 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Horn, R.A., Johnson, C.R.: Topics in Matrix Analysis. Cambridge University Press, Cambridge (1986)zbMATHGoogle Scholar
  15. 15.
    Ibe, O., Trivedi, K.: Stochastic Petri net models of polling systems. IEEE Journal on Selected Areas in Communications 8(9), 1649–1657 (1990)CrossRefGoogle Scholar
  16. 16.
    Jensen, A.: Markoff chains as an aid in the study of markoff processes. In: Skand. Aktuarietidskrift, vol. 36, pp. 87–91 (1953)Google Scholar
  17. 17.
    Katoen, J.-P., Zapreev, I.S.: Safe on-the-fly steady-state detection for time-bounded reachability. In: QEST, pp. 301–310. IEEE CS, Los Alamitos (2006)Google Scholar
  18. 18.
    Katoen, J.-P., Zapreev, I.S., Hahn, E.M., Hermanns, H., Jansen, D.N.: The ins and outs of the probabilistic model checker MRMC. In: Quantitative Evaluation of Systems, pp. 167–176. IEEE CS Press, Los Alamitos (2009)Google Scholar
  19. 19.
    Kwiatkowska, M., Norman, G., Parker, D.: Symmetry reduction for probabilistic model checking. In: Ball, T., Jones, R.B. (eds.) CAV 2006. LNCS, vol. 4144, pp. 234–248. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  20. 20.
    Massink, M., Katoen, J.-P., Latella, D.: Model checking dependability attributes of wireless group communication. In: Dependable Systems and Networks (DSN), pp. 711–720. IEEE CS Press, Los Alamitos (2004)Google Scholar
  21. 21.
    Moler, C., Van Loan, C.: Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later. SIAM Review 45(1), 3–49 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Rinehart, R.F.: The equivalence of definitions of a matrix function. The American Mathematical Monthly 62(6), 395–414 (1955)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Saad, Y.: Analysis of some Krylov subspace approximations to the matrix exponential operator. SIAM Journal on Numerical Analysis 29(1), 209–228 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Sidje, R.: Expokit: a software package for computing matrix exponentials. ACM Trans. Math. Softw. 24(1), 130–156 (1998)CrossRefzbMATHGoogle Scholar
  25. 25.
    Sidje, R., Stewart, W.J.: A survey of methods for computing large sparse matrix exponentials arising in Markov chains. Markov Chains, Computational Statistics and Data Analysis 29, 345–368 (1996)CrossRefzbMATHGoogle Scholar
  26. 26.
    Trefethen, L.N., Bau III, D.: Numerical Linear Algebra. Society for Industrial and Applied Mathematics, Philadelphia (1997)CrossRefzbMATHGoogle Scholar
  27. 27.
    van Moorsel, A., Sanders, W.: Adaptive uniformization. Communications in Statistics - Stochastic Models 10(3), 619–648 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Ward, R.C.: Numerical computation of the matrix exponential with accuracy estimate. SIAM Journal on Numerical Analysis 14(4), 600–610 (1977)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Weiss, R.: Error-minimizing Krylov subspace methods. SIAM Journal on Scientific Computing 15(3), 511–527 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Zapreev, I.S.: Model Checking Markov Chains: Techniques and Tools. PhD thesis, Univ. of Twente (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Falko Dulat
    • 1
  • Joost-Pieter Katoen
    • 1
  • Viet Yen Nguyen
    • 1
  1. 1.Software Modeling and Verification GroupRWTH Aachen UniversityGermany

Personalised recommendations