Abstract
Markovian models have been used for about a century now for the evaluation of the performance and dependability of computer and communication systems. In this paper, we give a concise overview of the most widely used classes of Markovian models, their solution and application.
After a brief introduction to performance and dependability evaluation in general, we introduce discrete-time Markov chains, continuous-time Markov chains and semi-Markov chains. Stepwisely, we develop the main equations that govern the steady-state and the transient behaviour of such Markov chains. We thereby emphasise on intuitively appealing explanations rather than on mathematical rigor. The relation between the various Markov chain types is explained in detail. Then, we discuss means to numerically solve the systems of linear equations (both direct and iterative ones) and the systems of differential equations that arise when solving for the steady-state and transient behaviour of Markovian models.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
A. Aziz, K. Sanwal, V. Singhal, and R. Brayton. Verifying continuous time Markov chains. In R. Alur and T. Henzinger, editors, Lecture Notes in Computer Science 1102, pages 269–276, 1996.
A. Aziz, V. Singhal, F. Balarin, R. Brayton, and A. Sangiovanni-Vincentelli. It usually works: the temporal logic of stochastic systems. In P. Wolper, editor, Lecture Notes in Computer Science 939, pages 155–165, 1995.
C. Baier, B.R. Haverkort, J.-P. Katoen, and H. Hermanns. On the logical characterisation of performability properties. In U. Montanari, J.D.P. Rolin, and E. Welzl, editors, Lecture Notes in Computer Science 1853, pages 780–792, 2000.
C. Baier, B.R. Haverkort, J.-P. Katoen, and H. Hermanns. Model checking continuous-time Markov chains by transient analysis. In E.A. Emerson and A.P. Sistla, editors, Lecture Notes in Computer Science 1855, pages 358–372, 2000.
C. Baier, J.-P. Katoen, and H. Hermanns. Approximate symbolic model checking of continuous-time Markov chains. In J.C.M. Baeten and S. Mauw, editors, Lecture Notes in Computer Science 1664, pages 146–161, 1999.
G. Balbo. Introduction to stochastic Petri nets. This volume.
A. Bell and B.R. Haverkort. Serial and parallel out-of-core solution of linear systems arising from generalised stochastic Petri net models. In Proceedings High Performance Computing 2001. Society for Computer Simulation, 2001.
M. Bernardo, R. Cleaveland, S. Sims, and W. Stewart. TwoTowers: a tool integrating functional and performance analysis of concurrent systems. In Proceedings FORTE/PSTV 1998, pages 457–467, 1998.
G. Bolch, S. Greiner, H. de Meer, and K.S. Trivedi. Queueing Networks and Markov Chains. John Wiley & Sons, 1998.
E. Brinksma and H. Hermanns. Process algebra and Markov chains. This volume.
S. Caselli, G. Conte, and P. Marenzoni. Parallel state space exploration for GSPN models. In G. De Michelis and M. Diaz, editors, Applications and Theory of Petri Nets 1995, volume 935 of Lecture Notes in Computer Science, pages 181–200. Springer-Verlag, 1995.
G. Chiola. A software package of the analysis of generalized stochastic Petri nets. In Proceedings of the 1st International Workshop on Timed Petri Nets, pages 136–143, Torino, Italy, July 1985. IEEE Computer Society Press.
G. Ciardo. Distributed and structured analysis approaches to study large and complex systems. This volume.
G. Ciardo, J. Gluckman, and D. Nicol. Distributed state space generation of discrete-state stochastic models. INFORMS Journal of Computing, 10(1):82–93, 1998.
G. Ciardo and A.S. Miner. Storage alternatives for large structured state spaces. In R. Marie, B. Plateau, M. Calzarossa, and G. Rubino, editors, Computer Performance Evaluation, Lecture Notes in Computer Science 1245, pages 44–57. Springer Verlag, 1997.
G. Ciardo, J. Muppala, and K.S. Trivedi. SPNP: Stochastic Petri net package. In Proceedings of the 3rd International Workshop on Petri Nets and Performance Models, pages 142–151. IEEE Computer Society Press, 1989.
E.M. Clarke, O. Grumberg, and D. Peled. Model checking. MIT Press, 1999.
N.M. van Dijk. Queueing Networks and Product Form: A Systems Approach. John Wiley & Sons, 1993.
E.M. Clarke, E.A. Emerson, and A.P. Sistla. Automatic verification of finite-state concurrent systems using temporal logic specifications. ACM Transactions on Programming Languages and Systems, 8(2):244–263, 1986.
A.K. Erlang. Solution of some problems in the theory of probabilities of significance in automatic telephone exchanges. The Post Office Electrical Engineer’s Journal, 10:189–197, 1917.
B.L. Fox and P.W. Glynn. Computing Poisson probabilities. Communications of the ACM, 31(4):440–445, 1988.
K.A. Frenkel. Allan L. Scherr-Big Blue’s time-sharing pioneer. Communications of the ACM, 30(10):824–828, 1987.
A. Goyal, S.S. Lavenberg, and K.S. Trivedi. The system availability estimator. Annals of Operations Research, 8:285–306, 1987.
W.K. Grassmann. Finding transient solutions in Markovian event systems through randomization. In W.J. Stewart, editor, Numerical Solution of Markov Chains, pages 357–371. Marcel Dekker, 1991.
D. Gross and D.R. Miller. The randomization technique as a modeling tool and solution procedure for transient Markov processes. Operations Research, 32(2):343–361, 1984.
A.L. Hageman and D.M. Young. Applied Iterative Methods. Academic Press, 1981.
B.R. Haverkort. Performance of Computer Communication Systems: A Model-Based Approach. John Wiley & Sons, 1998.
B.R. Haverkort, A. Bell, and H. Bohnenkamp. On the efficient sequential and distributed generation of very large Markov chains from stochastic Petri nets. In Proceedings of the 8th International Workshop on Petri Nets and Performance Models, pages 12–21. IEEE Computer Society Press, 1999.
B.R. Haverkort, R. Marie, G. Rubino, and K.S. Trivedi (editors). Performability Modelling: Techniques and Tools. John Wiley & Sons, 2001.
B.R. Haverkort and I.G. Niemegeers. Performability modelling tools and techniques. Performance Evaluation, 25:17–40, 1996.
B.R. Haverkort and K.S. Trivedi. Specification and generation of Markov reward models. Discrete-Event Dynamic Systems: Theory and Applications, 3:219–247, 1993.
H. Hermanns, U. Herzog, U. Klehmet, V. Mertsiotakis, and M. Siegle. Compositional performance modelling with the tipptool. Performance Evaluation, 39:5–35, 2000.
H. Hermanns, J.-P. Katoen, J. Meyer-Kayser, and M. Siegle. A Markov chain model checker. In S. Graf and M. Schwartzbach, editors, Lecture Notes in Computer Science 1785, pages 347–362. Springer-Verlag, 2000.
J. Hillston. A Compositional Approach to Performance Modelling. PhD thesis, University of Edinburgh, 1994.
J. Hillston. Exploiting structure in solution: decomposing compositional models. This volume.
A.S. Hornby. Oxford Advanced Learner’s Dictionary of Current English. Oxford University Press, 1974.
R.A. Howard. Dynamic Probabilistic Systems; Volume I: Markov models. John Wiley & Sons, 1971.
R.A. Howard. Dynamic Probabilistic Systems; Volume II: Semi-Markov and decision processes. John Wiley & Sons, 1971.
A. Jensen. Markov chains as an aid in the study of Markov processes. Skand. Aktuarietidskrift, 3:87–91, 1953.
K. Kant. Introduction to Computer System Performance Evaluation. McGraw-Hill, 1992.
J.-P. Katoen. Concepts, Algorithms and Tools for Model Checking. Universität Erlangen-Nürnberg, 1999.
J.G. Kemeny and J.L. Snell. Finite Markov chains. Van Nostrand, Princeton, 1960.
D.G. Kendall. Some problems in the theory of queues. Journal of the Royal Statistical Society, Ser. B, 13:151–185, 1951.
L. Kleinrock. Queueing Systems; Volume 1: Theory. John Wiley & Sons, 1975.
L. Kleinrock. Queueing Systems; Volume 2: Computer Applications. John Wiley & Sons, 1976.
A.N. Kolmogorov. Anfangsgrü;nde der Theorie der Markoffschen Ketten mit unendlichen vielen möglichen Zuständen. Mat. Sbornik N.S., pages 607–610, 1936.
U. Krieger, B. Müller-Clostermann, and M. Sczittnick. Modelling and analysis of communication systems based on computational methods for Markov chains. IEEE Journal on Selected Areas in Communications, 8(9):1630–1648, 1990.
V.G. Kulkarni. Modeling and Analysis of Stochastic Systems. Chapman & Hall, London, Glasgow, Weinheim, 1995.
A.A. Markov. Investigations of an important case of dependent trails. Izvestia Acad. Nauk VI, Series I (in Russian), 61, 1907.
J.F. Meyer. On evaluating the performability of degradable computing systems. IEEE Transactions on Computers, 29(8):720–731, 1980.
J.F. Meyer. Closed-form solutions of performability. IEEE Transactions on Computers, 31(7):648–657, 1982.
C. Moler and C.F. van Loan. Nineteen dubious ways to compute the exponential of a matrix. SIAM Review, 20(4):801–835, 1978.
A.P.A. van Moorsel. Performability Evaluation Concepts and Techniques. PhD thesis, University of Twente, 1993.
A.P.A. van Moorsel and B.R. Haverkort. Probabilistic evaluation for the analytical solution of large Markov models: Algorithms and tool support. Microelectronics and Reliability, 36(6):733–755, 1996.
A.P.A. van Moorsel and W.H. Sanders. Adaptive uniformization. Stochastic Models, 10(3):619–648, 1994.
B. Müller-Clostermann. NUMAS-a tool for the numerical analysis of computer systems. In D. Potier, editor, Proceedings of the International Conference on Modelling Techniques and Tools for Performance Analysis, pages 141–154. North-Holland, 1985.
J.K. Muppala and K.S. Trivedi. Numerical transient solution of finite Markovianqueueing systems. In U. Bhat, editor, Queueing and Related Models. Oxford UniversityPress, 1992
M.F. Neuts. Matrix Geometric Solutions in Stochastic Models: An Algorithmic Approach. Johns Hopkins University Press, 1981.
M.A. Qureshi and W.H. Sanders. A new methodology for calculating distributions of reward accumulated during a finite interval. In Proceedings of the 26th Symposium on Fault-Tolerant Computer Systems (Sendai, Japan, June 1996), pages 116–125. IEEE Computer Society Press, 1996.
W.H. Sanders and J.-F Meyer. Stochastic activity networks: Formal definitions and concepts. This volume.
W.H. Sanders and J.F. Meyer. Reduced-base model construction for stochastic activity networks. IEEE Journal on Selected Areas in Communications, 9(1):25–36, 1991.
W.H. Sanders, W.D. Obal, M.A. Qureshi, and F.K. Widnajarko. The UltraSAN modeling environment. Performance Evaluation, 24:89–115, 1995.
E. de Souza e Silva and H.R. Gail. Calculating availability and performability measures of repairable computer systems using randomization. Journal of the ACM, 36(1):171–193, 1989.
E. de Souza e Silva and H.R. Gail. Performability analysis of computer systems: from model specification to solution. Performance Evaluation, 1:157–196, 1992.
W.J. Stewart. MARCA: Markov chain analyzer. a software package for Markov modeling. In W.J. Stewart, editor, Numerical Solution of Markov Chains, pages 37–62. Marcel Dekker, 1991.
W.J. Stewart. Introduction to the Numerical Solution of Markov Chains. Princeton University Press, 1994.
K.S. Trivedi. Probability and Statistics with Reliability, Queueing and Computer Science Applications. Prentice-Hall, 1982.
M. Veran and D. Potier. QNAP2: A portable environment for queueing system modeling. In D. Potier, editor, Modelling Techniques and Tools for Computer Performance Evaluation, pages 25–63._North-Holland, 1984.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Haverkort, B.R. (2001). Markovian Models for Performance and Dependability Evaluation. In: Brinksma, E., Hermanns, H., Katoen, JP. (eds) Lectures on Formal Methods and PerformanceAnalysis. EEF School 2000. Lecture Notes in Computer Science, vol 2090. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44667-2_2
Download citation
DOI: https://doi.org/10.1007/3-540-44667-2_2
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42479-6
Online ISBN: 978-3-540-44667-5
eBook Packages: Springer Book Archive