Chinese Science Bulletin

, 44:97 | Cite as

Algorithms for Markov stochastic models

  • Guanghui Xu


A survey of the main results of algorithmic studies of Markov processes and related stochastic models is shown. It consists of stationary solutions, transient solutions, first passage times, and multidimensional denumerable state Markov processes. In conclusion, some remarks on further works are presented.


Markov press stochastic model algorithm stationary solutions transient solutions first passage time uniform error 


  1. 1.
    Tijms, H. C.,Stochastic Models: An Algorithmic Approach, New York: Wiley, 1994.Google Scholar
  2. 2.
    Grassmann, W. K., Computational methods in probability theory, inHandbooks in Operations Research and Management Science, Vo1..2,Stochastic Models, Amsterdam: North-Holland, 1990, 199–254.Google Scholar
  3. 3.
    Hsu, G. H.,Stochastic Service Systems (in Chinese), 2nd ed., Beijing: Science Press, 1988.Google Scholar
  4. 4.
    Neuts, M. F.,Matrix-Geometric Solutions in Stochastic Models, Baltimore: Johns Hopkins Univ. Press, 1981.Google Scholar
  5. 5.
    Neuts, M. F.,Structured Stochastic Matrices of M/ G/l Type and Their Applications, New York: Dekker, 1989.Google Scholar
  6. 6.
    Grassmann, W. K., Heyman, D. P., Computation of steady-state probabilities for infinite Markov chains with repeating rows,ORSA J. Comput., 1993, 5: 292.Google Scholar
  7. 7.
    Heyman, D. P., A decomposition theorem for infinite stochastic matrices,J. Appl. Prob., 1995, 32: 893.CrossRefGoogle Scholar
  8. 8.
    Bailey, N. T. J., A continuous time treatment of a simple queue using generating functions,J. Roy. Statist. Soc., 1954, B16: 288.Google Scholar
  9. 9.
    Conolly, B. W., A difference equation technique applied to the simple quene,J. Roy. Statist. Soc., 1958, B20: 165.Google Scholar
  10. 10.
    Cox, D. R., Smith, W. L.,Queues, London: Methuen & Co. Ltd., 1961.Google Scholar
  11. 11.
    Karlin, S., McGregor, J., Many-server queueing process with Poisson input and exponential senrice times,Pacific J. Math., 1958, 8: 87.Google Scholar
  12. 12.
    Yue, M. I., On the problemM/ M/s in the theory of queues,Acta Math. Sinica (in Chinese), 1959, 9: 494.Google Scholar
  13. 13.
    Saaty, T. L., Time-dependent solution of the many-server Poission queue,Opns. Res., 1960, 8: 755.Google Scholar
  14. 14.
    Takacs, L., The transient behavior of a single server queueing process with a Poisson input, inProc. Fourth Berkeley Symp. on Math. Statist. and Prob., Berkeley: University of California Press, 1961, 2: 535.Google Scholar
  15. 15.
    Conolly, B. W., A difference equation technique applied to the simple queue with arbitrary arrival interval distribution,J. Roy. Statist. Soc., 1958, B20: 168.Google Scholar
  16. 16.
    Takacs, L., Transient behavior of single-server queueing processes with recurrent input and exponential distributed service times,Opns. Res., 1960, 8: 231.Google Scholar
  17. 17.
    Wu, F., On theGI/M/n queueing process,Acta Math. Sinica (in Chinese), 1961, 11: 295.Google Scholar
  18. 18.
    Hsu, G. H., The transient behavior of the queueing processGI/M/n, Chinese Math., 1965, 6: 393.Google Scholar
  19. 19.
    Bhat, U. N., Transient behavior of multiserver queues with recurrent input and exponential service times,J. Appl. Prob., 1968, 5: 158.CrossRefGoogle Scholar
  20. 20.
    de Smit, J. H. A., On the many-server queue with exponential service times,Adv. Appl. Prob., 1973, 5: 170.CrossRefGoogle Scholar
  21. 21.
    Grassmann, W. K., Transient solutions in Markovian queueing systems,Comput. Opns. Res., 1977, 4: 47.CrossRefGoogle Scholar
  22. 22.
    Kohlas, J.,Stochastic Methods of Operations Research, London: Cambridge University Press, 1982.Google Scholar
  23. 23.
    Gross, D., Miller, D. R., The randomization technique as a modelling tool and solution procedure for transient Markov processes,Opns. Res., 1984, 32: 343.CrossRefGoogle Scholar
  24. 24.
    Reibman, A., Trivedi, K., Numerical transient analysis of Markov models,Comput. Opns. Res., 1988, 15: 19.CrossRefGoogle Scholar
  25. 25.
    Grassmann, W. K., Transient solutions in Markovian queues,Eur. J. Opns. Res., 1977, 1: 396.CrossRefGoogle Scholar
  26. 26.
    Zhang, J., Coyle, E. J., Transient analysis of quasi-birth-death processes,Stoch Models, 1989, 5: 459.CrossRefGoogle Scholar
  27. 27.
    Hsu, G. H., Yuan, X. M., Transient solutions for denumerable-state Markov processes,J. Appl. Prob., 1994, 31: 635.CrossRefGoogle Scholar
  28. 28.
    Ramaswami, V., The busy period of queues which have a matrix-geometric steady state probability vector,Opsearch, 1982, 9: 238.Google Scholar
  29. 29.
    Hsu, G. H., He, Q. M., The distribution of the first passage time for the Markov processes ofGI/M/1 type,Stoch. Models, 1991, 7: 397.CrossRefGoogle Scholar
  30. 30.
    Melamed, B., Yadin, M., Randomization procedures in the computation of comulative-time distributions over discrete state Markov processes,Opns. Res., 1984, 32: 926.Google Scholar
  31. 31.
    Melamed, B., Yadin, M., Numerical computation of sojourn-time distributions in queueing networks,JACM, 1984, 31: 839.CrossRefGoogle Scholar
  32. 32.
    Hsu, G. H., Yuan X. M., The first passage times and their algorithms for Makrov processes,Stoch. Models, 1995, 11: 195.CrossRefGoogle Scholar
  33. 33.
    Hsu, G. H., Yuan X. M., First passage times: busy periods and waiting times,Science in China, Ser. A, 1995, 38: 1187.Google Scholar
  34. 34.
    Latouche, G., Ramaswami, V., A logarithmic reduction algorithm for quasi-birth-death processes,J. Appl. Prob., 1993, 30: 650.CrossRefGoogle Scholar
  35. 35.
    Ramaswami, V., Taylor, P. G., Some properties of the rate operators in level dependent quasi-birth-and-death processes with a countable number of phases,Stoch. Models, 1996, 12: 143.CrossRefGoogle Scholar
  36. 36.
    Hsu, G. H., He, Q. M., Liu, X. S., The matched queueing system with a double input,Acta Math. Appl. Sinica, 1990, 13: 39.Google Scholar
  37. 37.
    Hsu, G. H., Xu, D. J., Transient solutions for multidimensional denumerable state Markov processes.,Queueing Syst., 1996, 23: 317.CrossRefGoogle Scholar
  38. 38.
    Hsu, G. H., Xu, D. J., First passage times for two-deimensional denumerable state Markov processes,Chinese Sci. Bulletin (in Chinese), 1997, 42(1): 9.Google Scholar
  39. 39.
    Hsu, G. H., Xu, D. J., First passage times for multidimensional denumerable state Markov processes,Chinese Science Bulletin (to appear).Google Scholar
  40. 40.
    Hsu, G. H. Jensen, U., The matched queueing networkPH/M/c→∘PH/PH/1, Queueing System, 1993, 13: 315.CrossRefGoogle Scholar

Copyright information

© Science in China Press 1999

Authors and Affiliations

  • Guanghui Xu
    • 1
  1. 1.Institute of Applied MathematicsChinese Academy of Sciences, Asian-Pacific Operations Research Center Within CAS and APORSBeijingChina

Personalised recommendations