Discrete-Time Markov-Reward Models of Production Systems

  • Ranga Mallubhatla
  • Krishna R. Pattipati
  • N. Viswanadham
Part of the The IMA Volumes in Mathematics and its Applications book series (IMA, volume 73)


In this paper we consider the discrete-time version of performability modeling. The discrete-time approach is well-suited for the performance studies of Automated Manufacturing Systems (AMS) in the presence of failures, repairs and reconfigurations. AMS exist in various configuration states and this transitional behavior is modeled using discrete-time Markov chains. In addition, the performance in each configuration state is modeled by a Markov reward structure, that is similar to the continuous-time versions. In this paper, we derive recursive expressions for the conditional densities and moments of the cumulative reward function, when the underlying Markov chain describing the evolution of the configuration states is homogenous. Examples are used to illustrate the methods obtained in the paper.


Buffer Size Material Handling System Mission Time Finite Buffer Irreducible Markov Chain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    K.R. Pattipati, R. Mallubhatla, V. Gopalakrishna AND N. Viswanatham, Markov-reward models and hyperbolic systems, Proceedings of Second International Workshop on Performability Modelling of Computer and Communication Systems, Le Mont Saint-Michel, France, June 1993.Google Scholar
  2. [2]
    L. Donatiello AND B.R. Iyer, Analysis of a composite performance reliability measure for fault-tolerant systems, Journal of the Association for Computing Machinery, vol. 34, January (1987), pp. 179–199.Google Scholar
  3. [3]
    E. de Souza, E Silva AND H.R. Gail, Calculating availability and performability measures of repairable computer systems using randomization, Journal of the Association for Computing Machinery, vol. 36, January (1989), pp. 171 –193.zbMATHGoogle Scholar
  4. [4]
    K.R. Pattipati AND S.A. Shah, On the computational aspects of performability models of fault-tolerant computer systems, IEEE Transactions on Computers, vol. 39, June (1990), pp. 832–836.Google Scholar
  5. [5]
    K.R. Pattipati, Y. Li, AND H.A.P. Blom, A unified framework for the performability evaluation of fault-tolerant computer systems, IEEE Transactions on Computers, vol. 42, No. 3, (1993), pp. 312–326.CrossRefGoogle Scholar
  6. [6]
    N. Viswanadham, Y. Narahari, AND R. Ram, Perform ability of automated manufacturing systems, Control and Dynamic Systems, vol. 47, (1991), pp. 77–120.Google Scholar
  7. [7]
    Y. Li, Analysis of Markov reward models of fault-tolerant computer systems, (M.S. Thesis) Department of Electrical and Systems Engineering, University of Connecticut, Storrs, CT 06269 - 3157, 1990.Google Scholar
  8. [8]
    J.F. Meyer, Performability modeling of distributed real time systems, Mathematical Computer Performance and Reliability, (G. Iazeolla, P.J. Courtois, AND A. Hordjik (eds.)) Elsevier Science Publishers (North Holland ) 1984.Google Scholar
  9. [9]
    M.D. Beudry, Performance-related reliability measures for computing systems, IEEE Trans. Comput., vol. C-27, June (1978), pp. 540–547.CrossRefGoogle Scholar
  10. [10]
    J.F. Meyer, on evaluating performability of degradable computing systems, .IEEE Trans. Comput, vol. C-29, no. 8, Aug (1980), pp. 720–731.CrossRefGoogle Scholar
  11. [11]
    B.R. Iyer, L. Donatiello, AND P. Heidelberger, Analysis of performability for stochastic models of fault-tolerant systems, IEEE Trans. Comput., vol. C-35, no. 10, Oct (1986), pp. 720–731.Google Scholar
  12. [12]
    V. Grassi, L. Donatiello, AND G. Iazeolla, Perform ability evaluation of multi- component fault-tolerant systems, IEEE Trans. Reliability, vol. 37, no. 2, June (1988), pp. 216–222.Google Scholar
  13. [13]
    R.M. Smith, K.S. Trivedi, AND A.V. Ramesh, Performability analysis: measures, an algorithm and a case study, IEEE Trans. Comput., vol. C-37, no. 4, Apr (1988), pp. 406–417.Google Scholar
  14. [14]
    E. de Souza E Sllva, AND H.R. Gail, Calculating cumulative operational time distributions of repairable computer systems, IEEE Trans. Comput., vol. C-35, no. 4, Apr (1986), pp. 322–332.Google Scholar
  15. [15]
    A. Goyal, AND A.N. Tantawi, A measure of gauranteed availability and its numerical evaluation, IEEE Trans. Comput., vol. C-37, no. 1, Jan (1988), pp. 25–32.Google Scholar
  16. [16]
    R.M. Smith, AND K.S. Trivedi, The analysisof computer systems using Markov reward models, in Stochastic Models of Computer and Communication Systems, (H. Takegi, ed.) Elsevier 1989.Google Scholar
  17. [17]
    A. Reibman, AND K.S. Trivedi, Transient analysis of cumulative measures of Markov model behavior, Stochastic Models, vol. 5, no. 4, (1989), pp. 683–710.MathSciNetzbMATHCrossRefGoogle Scholar
  18. [18]
    U. Sumita, J.G. Shantikumar AND Y. Masuda, Analysis of fault-tolerant computer systems, Microelectronics and Reliability, vol. 27, (1987), pp. 65–78.CrossRefGoogle Scholar
  19. [19]
    P.S. Puri, A method for studying the integral functionals of stochastic processes with applications: I the Markov chain case, Journal of Applied Probability, vol. 8, no. 2, June (1971), pp. 331–343.Google Scholar
  20. [20]
    A. Reibman, AND K.S. Trivedi, Numerical transient analysis of Markov models, Computers and Operations Research, vol. 15, no. 1, (1988), pp. 19–36.zbMATHCrossRefGoogle Scholar
  21. [21]
    G. Ciardo, R. Marie, B. Sericola, AND K.S. Trivedi, Performability analysis using semi-Markov reward processes, IEEE Trans, on Computers, vol. 39, No. 10, Oct (1990), pp. 1251–1264.Google Scholar
  22. [22]
    R. Huslende, A combined evaluation of performance and reliability for degradable systems, ACM/SIGMETRICS Conf. on Measurement and Modeling of Computer Systems, ACM, 1981, pp. 157–164.Google Scholar
  23. [23]
    V.G. Kulkarni, V.F. Nicola, AND K.S. Trivedi, On modeling the performance and reliability of multi-mode computer systems, The Journal of Systems and Software, 6(1 & 2), May (1986), pp. 175–183.Google Scholar
  24. [24]
    V.G. Kulkarni, V.F. Nicola, AND K.S. Trivedi, On completion time of a job on multi-mode systems, Advances in Applied Probability, Vol. 19, Dec (1987), pp. 932–954.Google Scholar
  25. [25]
    J. Meyer, Closed form solutions of performability, IEEE Trans, on Computers, Vol. 31, No. 7, July (1982), pp. 648–657.Google Scholar
  26. [26]
    W.K. Grassmann, Means and variances of time averages in Markovian environments, European Journal of Operations Research, Vol. 31, No. 1, (1987), pp. 132–139.CrossRefGoogle Scholar
  27. [27]
    Jean-Luc Deleersnyder, T.J. Hodgson, H. Muller, AND P.J. O’Grady, Kanban controlled pull systems: an analytic approach, Management Science, vol. 35, No. 9, September (1989), pp. 1079–1091.Google Scholar
  28. [28]
    S.B. Gershwin, Variance of Output of a Tandem Production System, Department of Mechanical Engineering, M.I.T. 1992.Google Scholar
  29. [29]
    G.J. Mlltenburg, Variance of the number of units produced on a transfer line with buffer inventories during a period of length T, Naval Research Logistics Quarterly, vol. 34, (1987), pp. 811–822.zbMATHCrossRefGoogle Scholar
  30. [30]
    S.B. Gershwin, AND I.E. Schick, Modeling and analysis of three-stage transfer lines with unreliable machines and finite buffers, Operations Research (1983), pp. 354–380.Google Scholar
  31. [31]
    R. Mallubhatla AND K.R. Pattipati, Discrete-time Markov-reward models: random rewards, in Proceedings of the Rensselaer’s Fourth International Conference on Computer Integrated Manufacturing and Automation Technology 1994.Google Scholar
  32. [32]
    R. Mallubhatla, K.R. Pattipati, AND N. Viswanadham, Moment recursions of the cumulative performance of production systems using discrete-time Markov reward models, in Proceedings of the IEEE International Conference on Robotics and Automation 1994.Google Scholar
  33. [33]
    R. Mallubhatla, K.R. Pattipati, AND N. Viswanadham, Discrete-time Markov-reward models of production systems, Technical Report TR93-2, Electrical and Systems Engineering Department, University of Connecticut, August 1993.Google Scholar
  34. [34]
    A. Papoulis, Probability, Random Variables, and Stochastic Processes, New York, McGraw-Hill 1965.zbMATHGoogle Scholar

Copyright information

© Springer-Verlag New York, Inc 1995

Authors and Affiliations

  • Ranga Mallubhatla
    • 1
  • Krishna R. Pattipati
    • 1
  • N. Viswanadham
    • 2
  1. 1.Department of Electrical and Systems EngineeringUniversity of ConnecticutStorrsUSA
  2. 2.Department of Computer Science and AutomationIndian Institute of ScienceBangaloreIndia

Personalised recommendations