Skip to main content
SpringerLink
Log in

Algorithms for Markov stochastic models

  • Reviews
  • Published:
Chinese Science Bulletin

Abstract

A survey of the main results of algorithmic studies of Markov processes and related stochastic models is shown. It consists of stationary solutions, transient solutions, first passage times, and multidimensional denumerable state Markov processes. In conclusion, some remarks on further works are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Tijms, H. C.,Stochastic Models: An Algorithmic Approach, New York: Wiley, 1994.

    Google Scholar 

  2. Grassmann, W. K., Computational methods in probability theory, inHandbooks in Operations Research and Management Science, Vo1..2,Stochastic Models, Amsterdam: North-Holland, 1990, 199–254.

    Google Scholar 

  3. Hsu, G. H.,Stochastic Service Systems (in Chinese), 2nd ed., Beijing: Science Press, 1988.

    Google Scholar 

  4. Neuts, M. F.,Matrix-Geometric Solutions in Stochastic Models, Baltimore: Johns Hopkins Univ. Press, 1981.

    Google Scholar 

  5. Neuts, M. F.,Structured Stochastic Matrices of M/ G/l Type and Their Applications, New York: Dekker, 1989.

    Google Scholar 

  6. Grassmann, W. K., Heyman, D. P., Computation of steady-state probabilities for infinite Markov chains with repeating rows,ORSA J. Comput., 1993, 5: 292.

    Google Scholar 

  7. Heyman, D. P., A decomposition theorem for infinite stochastic matrices,J. Appl. Prob., 1995, 32: 893.

    Article  Google Scholar 

  8. Bailey, N. T. J., A continuous time treatment of a simple queue using generating functions,J. Roy. Statist. Soc., 1954, B16: 288.

    Google Scholar 

  9. Conolly, B. W., A difference equation technique applied to the simple quene,J. Roy. Statist. Soc., 1958, B20: 165.

    Google Scholar 

  10. Cox, D. R., Smith, W. L.,Queues, London: Methuen & Co. Ltd., 1961.

    Google Scholar 

  11. Karlin, S., McGregor, J., Many-server queueing process with Poisson input and exponential senrice times,Pacific J. Math., 1958, 8: 87.

    Google Scholar 

  12. Yue, M. I., On the problemM/ M/s in the theory of queues,Acta Math. Sinica (in Chinese), 1959, 9: 494.

    Google Scholar 

  13. Saaty, T. L., Time-dependent solution of the many-server Poission queue,Opns. Res., 1960, 8: 755.

    Google Scholar 

  14. Takacs, L., The transient behavior of a single server queueing process with a Poisson input, inProc. Fourth Berkeley Symp. on Math. Statist. and Prob., Berkeley: University of California Press, 1961, 2: 535.

    Google Scholar 

  15. Conolly, B. W., A difference equation technique applied to the simple queue with arbitrary arrival interval distribution,J. Roy. Statist. Soc., 1958, B20: 168.

    Google Scholar 

  16. Takacs, L., Transient behavior of single-server queueing processes with recurrent input and exponential distributed service times,Opns. Res., 1960, 8: 231.

    Google Scholar 

  17. Wu, F., On theGI/M/n queueing process,Acta Math. Sinica (in Chinese), 1961, 11: 295.

    Google Scholar 

  18. Hsu, G. H., The transient behavior of the queueing processGI/M/n, Chinese Math., 1965, 6: 393.

    Google Scholar 

  19. Bhat, U. N., Transient behavior of multiserver queues with recurrent input and exponential service times,J. Appl. Prob., 1968, 5: 158.

    Article  Google Scholar 

  20. de Smit, J. H. A., On the many-server queue with exponential service times,Adv. Appl. Prob., 1973, 5: 170.

    Article  Google Scholar 

  21. Grassmann, W. K., Transient solutions in Markovian queueing systems,Comput. Opns. Res., 1977, 4: 47.

    Article  Google Scholar 

  22. Kohlas, J.,Stochastic Methods of Operations Research, London: Cambridge University Press, 1982.

    Google Scholar 

  23. Gross, D., Miller, D. R., The randomization technique as a modelling tool and solution procedure for transient Markov processes,Opns. Res., 1984, 32: 343.

    Article  Google Scholar 

  24. Reibman, A., Trivedi, K., Numerical transient analysis of Markov models,Comput. Opns. Res., 1988, 15: 19.

    Article  Google Scholar 

  25. Grassmann, W. K., Transient solutions in Markovian queues,Eur. J. Opns. Res., 1977, 1: 396.

    Article  Google Scholar 

  26. Zhang, J., Coyle, E. J., Transient analysis of quasi-birth-death processes,Stoch Models, 1989, 5: 459.

    Article  Google Scholar 

  27. Hsu, G. H., Yuan, X. M., Transient solutions for denumerable-state Markov processes,J. Appl. Prob., 1994, 31: 635.

    Article  Google Scholar 

  28. Ramaswami, V., The busy period of queues which have a matrix-geometric steady state probability vector,Opsearch, 1982, 9: 238.

    Google Scholar 

  29. Hsu, G. H., He, Q. M., The distribution of the first passage time for the Markov processes ofGI/M/1 type,Stoch. Models, 1991, 7: 397.

    Article  Google Scholar 

  30. Melamed, B., Yadin, M., Randomization procedures in the computation of comulative-time distributions over discrete state Markov processes,Opns. Res., 1984, 32: 926.

    Google Scholar 

  31. Melamed, B., Yadin, M., Numerical computation of sojourn-time distributions in queueing networks,JACM, 1984, 31: 839.

    Article  Google Scholar 

  32. Hsu, G. H., Yuan X. M., The first passage times and their algorithms for Makrov processes,Stoch. Models, 1995, 11: 195.

    Article  Google Scholar 

  33. Hsu, G. H., Yuan X. M., First passage times: busy periods and waiting times,Science in China, Ser. A, 1995, 38: 1187.

    Google Scholar 

  34. Latouche, G., Ramaswami, V., A logarithmic reduction algorithm for quasi-birth-death processes,J. Appl. Prob., 1993, 30: 650.

    Article  Google Scholar 

  35. Ramaswami, V., Taylor, P. G., Some properties of the rate operators in level dependent quasi-birth-and-death processes with a countable number of phases,Stoch. Models, 1996, 12: 143.

    Article  Google Scholar 

  36. Hsu, G. H., He, Q. M., Liu, X. S., The matched queueing system with a double input,Acta Math. Appl. Sinica, 1990, 13: 39.

    Google Scholar 

  37. Hsu, G. H., Xu, D. J., Transient solutions for multidimensional denumerable state Markov processes.,Queueing Syst., 1996, 23: 317.

    Article  Google Scholar 

  38. Hsu, G. H., Xu, D. J., First passage times for two-deimensional denumerable state Markov processes,Chinese Sci. Bulletin (in Chinese), 1997, 42(1): 9.

    Google Scholar 

  39. Hsu, G. H., Xu, D. J., First passage times for multidimensional denumerable state Markov processes,Chinese Science Bulletin (to appear).

  40. Hsu, G. H. Jensen, U., The matched queueing networkPH/M/c→∘PH/PH/1, Queueing System, 1993, 13: 315.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Applied Mathematics, Chinese Academy of Sciences, Asian-Pacific Operations Research Center Within CAS and APORS, 100080, Beijing, China

    Guanghui Xu

Authors
  1. Guanghui Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

About this article

Cite this article

Xu, G. Algorithms for Markov stochastic models. Chin. Sci. Bull. 44, 97–100 (1999). https://doi.org/10.1007/BF02884726

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02884726

Keywords

Search

Navigation