Skip to main content
SpringerLink
Log in

Automated Generation and Analysis of Markov Reward Models Using Stochastic Reward Nets

  • Conference paper
Linear Algebra, Markov Chains, and Queueing Models

Part of the book series: The IMA Volumes in Mathematics and its Applications ((IMA,volume 48))

Abstract

Markov and Markov reward models are widely used for the performance and reliability analysis of computer and communication systems. Models of real systems often contain thousands or even millions of states. We propose the use of Stochastic Reward Nets (SRNs) for the automatic generation of these large Markov reward models. SRNs do allow the concise specification of practical performance, reliability and per-formability models.

An added advantage of using SRNs lies in the possibility of analyzing the (time-independent) logical behavior of the modeled system. This helps both the validation of the system (is the right system being built?) and of the model (does the model correctly represent the system?).

We discuss the methods to convert SRNs into Markov reward processes automatically. We review the solution techniques for the steady state and transient analysis of SRNs and Markov reward processes. We also discuss methods for the sensitivity analysis of SRNs.

This work was supported in part by the National Science Foundation under Grant CCR-9108114 and by the Naval Surface Weapons Center under the ONR Grant N00014-91-5-4162.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. A. V. Aho, J. E. Hopcroft, and J. D. Ullman. The Design and Analysis of Computer Algorithms. Addison-Wesley, Menlo Park, CA, USA, 1974.

    MATH  Google Scholar 

  2. M. Ajmone Marsan, G. Balbo, A. Bobbio, G. Chiola, G. Conte, and A. Cumani. The effect of execution policies on the semantics and analyis of Stochastic Petri Nets. IEEE Transactions on Software Engineering, 15(7):832–846, July 1989.

    Article  MathSciNet  Google Scholar 

  3. M. Ajmone Marsan, G. Balbo, and G. Conte. A class of Generalized Stochastic Petri Nets for the performance evaluation of multiprocessor systems. ACM Transactions on Computer Systems, 2(2):93–122, May 1984.

    Article  Google Scholar 

  4. G. Balbo, G. Chiola, G. Franceschinis, and G. Molinari Roet. On the efficient construction of the tangible reachability graph of generalized stochastic Petri nets. In Proceedings of the IEEE International Workshop on Petri Nets and Performance Models, Madison, WI, USA, Aug. 1987.

    Google Scholar 

  5. F. Baskett, K. M. Chandy, R. R. Muntz, and F. Palacios-Gomez. Open, Closed, and Mixed networks of queues with different classes of customers. Journal of the ACM, 22(2):335–381, Apr. 1975.

    Article  Google Scholar 

  6. M. D. Beaudry. Performance-related reliability measures for computing systems. IEEE Transactions on Computers, C-27(6):540–547, June 1978.

    Article  Google Scholar 

  7. J. Bechta Dugan, K. S. Trivedi, R. M. Geist, and V. F. Nicola. Extended Stochastic Petri Nets: applications and analysis. In E. Gelenbe, editor, Performance ’84, North-Holland, Amsterdam, 1985, pages 507–519.

    Google Scholar 

  8. J. Bechta Dugan, K. S. Trivedi, M. K. Smotherman, and R. M. Geist. The Hybrid Automated Reliability Predictor. AIAA Journal of Guidance, Control and Dynamics, 9(3):319–331, May 1986.

    Article  MATH  Google Scholar 

  9. J. T. Blake, A. L. Reibman, and K. S. Trivedi. Sensitivity analysis of reliability and performance measures for multiprocessor systems. In Proceedings of the 198 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, Santa Fe, NM, USA, pages 177–186, May 1988.

    Chapter  Google Scholar 

  10. G. Chiola. A software package for the analysis of Generalized Stochastic Petri Net models. In Proceedings of the IEEE International Workshop on Timed Petri Nets, Torino, Italy, July 1985.

    Google Scholar 

  11. H. Choi and K. S. Trivedi. Approximate Performance Models of Polling Systems using Stochastic Petri Nets. In Proceedings of the IEEE INFOCOM 92, Florence, Italy, May 1992.

    Google Scholar 

  12. G. Ciardo. Analysis of large stochastic Petri net models. PhD thesis, Duke University, Durham, NC, USA, 1989.

    Google Scholar 

  13. G. Ciardo, R. A. Marie, B. Sericola, and K. S. Trivedi. Performability analysis using semi-Markov reward processes. IEEE Transactions on Computers, 39(10):1251–1264, Oct. 1990.

    Article  Google Scholar 

  14. G. Ciardo, J. Muppala, and K. S. Trivedi. Analyzing concurrent and fault-tolerant software using stochastic Petri nets. Journal of Parallel and Distributed Computing. To appear.

    Google Scholar 

  15. G. Ciardo, J. Muppala, and K. S. Trivedi. On the solution of GSPN reward models. Performance Evaluation, 12(4):237–253, 1991.

    Article  MATH  Google Scholar 

  16. G. Ciardo and K. S. Trivedi. A decomposition approach for stochastic Petri net models. In Proceedings of the Fourth IEEE International Workshop on Petri Nets and Performance Models (PNPM91), Melbourne, Australia, pages 74–83, Dec. 1991.

    Chapter  Google Scholar 

  17. G. Ciardo, K. S. Trivedi, and J. Muppala. SPNP: stochastic Petri net package. In Proceedings of the Third IEEE International Workshop on Petri Nets and Performance Models (PNPM89), Kyoto, Japan, pages 142 – 151, Dec. 1989.

    Google Scholar 

  18. A. Cumani. ESP — A package for the evaluation of stochastic Petri nets with phase-type distributed transitions times. In Proceedings of the IEEE International Workshop on Timed Petri Nets, Torino, Italy, July 1985.

    Google Scholar 

  19. G. Dahlquist and A. Björck. Numerical Methods. Prentice-Hall, Englewood Cliffs, N.J., 1974.

    Google Scholar 

  20. E. de Souza e Silva and H. R. Gail. Calculating availability and performability measures of repairable computer systems using randomization. J. A CM., 36(1):171–193, Jan. 1989.

    MATH  Google Scholar 

  21. E. de Souza e Silva and R. R. Muntz. Queueing networks: solutions and applications. In H. Takagi, editor, Stochastic Analysis of Computer and Communication Systems. Elsevier Science Publishers B.V. (North-Holland), pages 319–400, 1990.

    Google Scholar 

  22. B. L. Fox and P. W. Glynn. Computing poisson probabilities. Commun. ACM., 31(4):440–445,Apr. 1988.

    Article  MathSciNet  Google Scholar 

  23. P. M. Frank. Introduction to System Sensitivity. Academic Press, New York, NY, 1978.

    MATH  Google Scholar 

  24. A. Goyal, W. C. Carter, E. de Souza e Silva, S. S. Lavenberg, and K. S. Trivedi. The System Availability Estimator. In Proceedings of the Sixteenth International Symposium on Fault-Tolerant Computing, pages 84–89, Vienna, Austria, July 1986.

    Google Scholar 

  25. A. Goyal, S. Lavenberg, and K. S. Trivedi. Probabilistic modeling of computer system availability. Annals of Operations Research, 8:285–306, Mar. 1987.

    Article  Google Scholar 

  26. W. K. Grassmann. Means and variances of time averages in Markovian environments. Eur. J. Oper. Res., 31(1):132–139, 1987.

    Article  MathSciNet  MATH  Google Scholar 

  27. D. Gross and D. Miller. The randomization technique as a modeling tool and solution procedure for transient Markov processes. Oper. Res., 32(2):926–944, Mar.-Apr. 1984.

    Article  MathSciNet  Google Scholar 

  28. P. J. Haas and G. S. Shedler. Stochastic Petri net representation of discrete event simulations. IEEE Transactions on Software Engineering, 15(4):381–393, Apr. 1989.

    Article  MathSciNet  MATH  Google Scholar 

  29. P. Heidelberger and A. Goyal. Sensitivity analysis of continuous time Markov chains using uniformization. In P. J. Courtois, G. Iazeolla, and O.J. Boxma, editors, Computer Performance and Reliability, North-Holland, Amsterdam, pages 93–104, 1988.

    Google Scholar 

  30. R. A. Howard. Dynamic Probabilistic Systems, Volume II: Semi-Markov and Decision Processes. John Wiley and Sons, New York, NY, 1971.

    Google Scholar 

  31. O. C. Ibe, H. Choi, and K. S. Trivedi. Performance Evaluation of Client-Server Systems. IEEE Transactions on Parallel and Distributed Systems, to appear.

    Google Scholar 

  32. O. C. Ibe, R. C. Howe, and K. S. Trivedi. Approximate availability analysis of vaxcluster systems. IEEE Trans. Reliability, R-38(l):146–152, Apr. 1989.

    Article  Google Scholar 

  33. O. C. Ibe and K. S. Trivedi. Stochastic Petri net models of polling systems. IEEE Journal on Selected Areas in Communications, 8(9):1649–1657, Dec. 1990.

    Article  Google Scholar 

  34. A. Jensen. Markoff chains as an aid in the study of Markoff processes. Skand. Aktuarietidskr., 36:87–91, 1953.

    MathSciNet  Google Scholar 

  35. K. Jensen. Coloured Petri nets and the invariant method. Theoretical Computer Science, 14:317–336, 1981.

    Article  MathSciNet  MATH  Google Scholar 

  36. H. Kantz and K. S. Trivedi. Reliability Modeling of the MARS System: A Case Study in the Use of Different Tools and Techniques. In Proceedings of the Fourth IEEE International Workshop on Petri Nets and Performance Models (PNPM91), Melbourne, Australia, Dec. 1991.

    Google Scholar 

  37. J. Keilson. Markov Chain Models — Rarity and Exponentiality. Applied Mathematical Sciences Ser. Vol. 28. Springer-Verlag, 1979.

    Book  MATH  Google Scholar 

  38. F. P. Kelly. Reversibility and Stochastic Networks. Wiley, 1979.

    MATH  Google Scholar 

  39. W. Kleinoder. Evaluation of task structures for hierarchical multiprocessor systems. In D. Potier, editor, Modelling Techniques and Tools for Performance Analysis. Elsevier Science Publishers B.V. (North Holland), 1985.

    Google Scholar 

  40. S. S. Lavenberg, editor. Computer Performance Modeling Handbook. Academic Press, New York, 1983.

    MATH  Google Scholar 

  41. E. D. Lazowska, J. Zahorjan, G. S. Graham, and K. C. Sevcick. Quantitative System Performance. Prentice-Hall, Englewood Cliffs, NJ, USA, 1984.

    Google Scholar 

  42. V. W. Mak and S. F. Lundstrom. Predicting performance of parallel computations. IEEE Transactions on Parallel and Distributed Systems, 1(3):257–270, July 1990.

    Article  Google Scholar 

  43. M. Malhotra and K. S. Trivedi. Higher order methods for the transient analysis of Markov chains. In Proc. Int. Conf. on the Performance of Distributed Systems and Integrated Communication Networks, Kyoto, Japan, Sept. 1991.

    Google Scholar 

  44. J. F. Meyer. Performability: a retrospective and some pointers to the future. Performance Evaluation, 14(3–4):139–156, 1992.

    Article  MATH  Google Scholar 

  45. C. Moler and C. F. V. Loan. Nineteen dubious ways to compute the exponential of a matrix. SIAM Review, 20(4):801–835, Oct. 1978.

    Article  MathSciNet  MATH  Google Scholar 

  46. M. K. Molloy. On the integration of delay and throughput measures in distributed processing models. PhD thesis, UCLA, Los Angeles, CA, USA, 1981.

    Google Scholar 

  47. J. K. Muppala. Performance and Dependability Modeling Using Stochastic Reward Nets. PhD thesis, Department of Electrical Engineering, Duke University, Durham, NC, Apr. 1991.

    Google Scholar 

  48. J. K. Muppala, A. S. Sathaye, R. C. Howe, and K. S. Trivedi. Dependability modeling of a heterogenous VAXcluster system using stochastic reward nets. in: Hardware and Software Fault Tolerance in Parallel Computing Systems, D. Averesky (ed.), Ellis Horwood Ltd., 1992. to appear.

    Google Scholar 

  49. J. K. Muppala, S. P. Woolet, and K. S. Trivedi. Real-time systems performance in the presence of failures. IEEE Computer, 24(5):37–47, May 1991.

    Google Scholar 

  50. T. Murata. Petri Nets: properties, analysis and applications. Proceedings of the IEEE, 77(4):541–579, Apr. 1989.

    Article  Google Scholar 

  51. J. L. Peterson. Petri Net Theory and the Modeling of Systems. Prentice-Hall, Englewood Cliffs, NJ, USA, 1981.

    Google Scholar 

  52. C. Petri. Kommunikation mit Automaten. PhD thesis, University of Bonn, Bonn, West Germany, 1962.

    Google Scholar 

  53. S. Pissanetzky. Sparse Matrix Technology. Academic Press, Orlando, FL, USA, 1984.

    MATH  Google Scholar 

  54. A. L. Reibman and K. S. Trivedi. Numerical transient analysis of Markov models. Computers and Operations Research, 15(1):19–36, 1988.

    Article  MATH  Google Scholar 

  55. A. L. Reibman and K. S. Trivedi. Transient analysis of cumulative measures of Markov model behavior. Stochastic Models, 5(4):683–710, 1989.

    Article  MathSciNet  MATH  Google Scholar 

  56. W. Reisig. Petri Nets, volume 4 of EATC Monographs on Theoretical Computer Science. Springer-Verlag, New York, 1985.

    MATH  Google Scholar 

  57. G. Rozenberg. Introduction to Petri nets, Dec. 1991. Tutorial Notes for the Fourth IEEE International Workshop on Petri Nets and Performance Models (PNPM91).

    Google Scholar 

  58. R. A. Sahner and K. S. Trivedi. Performance and reliability analysis using directed acyclic graphs. IEEE Transactions on Software Engineering, SE-14(10):1105–1114, Oct. 1987.

    Article  Google Scholar 

  59. R. A. Sahner and K. S. Trivedi. Reliability modeling using SHARPE. IEEE Transactions on Reliability, R-36(2):186–193, June 1987.

    Article  Google Scholar 

  60. W. H. Sanders and J. F. Meyer. METASAN: a performability evaluation tool based on Stochastic Activity Networks. In Proceedings of the ACM-IEEE Comp. Soc. Fall Joint Comp. Conf., Nov. 1986.

    Google Scholar 

  61. M. L. Shooman. Probabilistic Reliability: An Engineering Approach. McGraw-Hill, New York, 1968.

    Google Scholar 

  62. J. Sifakis. Use of Petri nets for performance evaluation. In H. Beilner and E. Gelenbe, editors, Measuring, Modelling, and Evaluating Computer Systems, pages 75–93. North Holland, 1977.

    Google Scholar 

  63. R. M. Smith and K. S. Trivedi. The analysis of computer systems using Markov reward processes. In H. Takagi, editor, Stochastic Analysis of Computer and Communication Systems. Elsevier Science Publishers B.V. (North-Holland), 1990, pages 589–629.

    Google Scholar 

  64. W. Stewart and A. Goyal. Matrix methods in large dependability models. Technical Report RC-11485, IBM T.J. Watson Res. Center, Yorktown Heights, NY, 10598, Nov. 1985.

    Google Scholar 

  65. D. P. Stotts and P. Godfrey. Place/transition nets with debit arcs. Information Processing Letters, 41:25–33, Jan. 1992.

    Article  MathSciNet  MATH  Google Scholar 

  66. G. Strang. Introduction to Applied Mathematics. Wellesley-Cambridge Press, Wellesley Massachusetts 02182, 1986.

    MATH  Google Scholar 

  67. H. Tardif, K. S. Trivedi, and A. V. Ramesh. Closed-form transient analysis of Markov chains. Technical Report CS-1988, Dept. of Computer Science, Duke University, Durham, NC, 27706, June 1988.

    Google Scholar 

  68. L. Tomek and K. S. Trivedi. Fixed-Point Iteration in Availability Modeling. In M. Dal Cin, editor, Informatik-Fachberichte, Vol. 91: Fehlertolerierende Rechensysteme, pages 229–240, Springer-Verlag, Berlin, 1991.

    Google Scholar 

  69. K. S. Trivedi. Probability & Statistics with Reliability, Queueing, and Computer Science Applications. Prentice-Hall, Englewood Cliffs, NJ, USA, 1982.

    Google Scholar 

  70. K. S. Trivedi, J. K. Muppala, S. P. Woolet, and B. R. Haverkort. Composite performance and dependability analysis. Perf. Eval., 14(3–4):197–215, 1992.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, College of William and Mary, P. O. Box 8795, 23815, Williamsburg, VA, USA

    Gianfranco Ciardo & Jogesh K. Muppala

  2. Dept. of Computer Science, University of Maryland, 20742, College Park, MD, USA

    Alex Blakemore

  3. IBM Corporation, Research Triangle Park, NC, 27709, USA

    Philip F. Chimento Jr.

  4. Dept. of Electrical Engineering, Duke University, Durham, NC, 27706, USA

    Kishor S. Trivedi

Authors
  1. Gianfranco Ciardo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alex Blakemore
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Philip F. Chimento Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jogesh K. Muppala
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kishor S. Trivedi
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Mathematics Department, North Carolina State University, 27695-8205, Raleigh, NC, USA

    Carl D. Meyer

  2. Department of Mathematics and Computer Science, Wake Forest University, 27109, Winston-Salem, NC, USA

    Robert J. Plemmons

Rights and permissions

Reprints and permissions

Copyright information

© 1993 Springer-Verlag New York, Inc.

About this paper

Cite this paper

Ciardo, G., Blakemore, A., Chimento, P.F., Muppala, J.K., Trivedi, K.S. (1993). Automated Generation and Analysis of Markov Reward Models Using Stochastic Reward Nets. In: Meyer, C.D., Plemmons, R.J. (eds) Linear Algebra, Markov Chains, and Queueing Models. The IMA Volumes in Mathematics and its Applications, vol 48. Springer, New York, NY. https://doi.org/10.1007/978-1-4613-8351-2_11

Download citation

  • DOI: https://doi.org/10.1007/978-1-4613-8351-2_11

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4613-8353-6

  • Online ISBN: 978-1-4613-8351-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation