Skip to main content
SpringerLink
Log in
Book cover

International Conference on Reliability and Statistics in Transportation and Communication

RelStat 2019: Reliability and Statistics in Transportation and Communication pp 100–109Cite as

Markov-Modulated Processes, Their Applications and Big Data Cases: State of the Art

  • Conference paper
  • First Online:

  • 985 Accesses

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 117))

Abstract

If the process is modulated (also driven or controlled) by the underlying Markov process, then such a process is called a Markov-modulated process. The study begins with an overview of Markov-additive processes, the earliest models found in the literature, which in turn are generalizations of Lévy processes. A Markov-additive process is a double-layer Markov process \( \left( {X, J} \right) \), where component X is additive and conditional increment depends only on Markov component \( J \). Depending on the type of model by means of which the component X is described, processes such as Poisson, Bernoulli, regression and other are considered. In many applications component J represents some non-observable extraneous factors, that’s why some authors call Hidden Markov Models. The study includes an attempt to systematize various Markov-modulated models, draw a boundary between different types, or vice versa to combine different models with the Markov component into one.

Various areas of application of this type of model are described, such as traffic, queuing theory, risk theory, management, health, communications etc. Examples are considered that illustrate the use of Markov-modulated models in the case of big data.

The research strategy includes a search for literature in the most popular and influential databases, such as ScienceDirect, Web of Science and Springerlink.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Pacheco, A., Tang, L.C., Prabhu, N.U.: Markov-Modulated Processes & Semiregenerative Phenomena. World Scientific, New Jersey (2009)

    MATH  Google Scholar 

  2. Çinlar, E.: Markov additive processes. I. Probab. Theory Relat. Fields 24(2), 85–93 (1972)

    MathSciNet  MATH  Google Scholar 

  3. Çinlar, E.: Markov additive processes. II. Probab. Theory Relat. Fields 24(2), 95–121 (1972)

    MathSciNet  MATH  Google Scholar 

  4. Ezhov, I.I., Skorokhod, A.V.: Markov processes with homogeneous second component. I. Theory of Probab. Appl. 14(1), 1–13 (1969)

    Article  MATH  Google Scholar 

  5. Arjas, E., Speed, T.P.: Topics in Markov additive processes. Math. Scand. 33(1), 171–192 (1974)

    MathSciNet  MATH  Google Scholar 

  6. Ney, P., Nummelin, E.: Markov additive processes I. Eigenvalue properties and limit theorems. The Ann. Probab. 561–592 (1987)

    Google Scholar 

  7. Ney, P., Nummelin, E.: Markov additive processes II. Large deviations. Ann. Probab. 15(2), 593–609 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bladt, M., Nielsen, B.F.: Markov additive processes. In: Matrix-Exponential Distributions in Applied Probability. Probability Theory and Stochastic Modelling, vol. 81. Springer, Boston (2017)

    Google Scholar 

  9. Ivanovs, J.: Splitting and time reversal for Markov additive processes. Stoch. Process. Their Appl. 127(8), 2699–2724 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  10. Andronov, A.M., Vishnevsky, V.M.: Reliability of two communication channels in a random environment. In: International Conference on Distributed Computer and Communication Networks. Springer, Cham (2018)

    Google Scholar 

  11. Dieker, A.B., Mandjes, M.: Extremes of Markov-additive processes with one-sided jumps, with queueing applications. Methodol. Comput. Appl. Probab. 13(2), 221–267 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  12. Feng, R., Shimizu, Y.: Potential measures for spectrally negative Markov additive processes with applications in ruin theory. Insur. Math. Econ. 59, 11–26 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  13. Ivanovs, J.: Markov-modulated Brownian motion with two reflecting barriers. J. Appl. Probab. 47(4), 1034–1047 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  14. Asmussen, S.: Stationary distributions for fluid flow models with or without Brownian noise. Commun. Stat. Stoch. Models 11(1), 21–49 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  15. Karandikar, R.L., Kulkarni, V.G.: Second-order fluid flow models: reflected Brownian motion in a random environment. Oper. Res. 43(1), 77–88 (1995)

    Article  MATH  Google Scholar 

  16. Rogers, L.C.G.: Fluid models in queueing theory and Wiener-Hopf factorization of Markov chains. Ann. Appl. Probab. 4(2), 390–413 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  17. Breuer, L.: Occupation times for Markov-modulated Brownian motion. J. Appl. Probab. 49(2), 549–565 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  18. Elliott, R.J., Swishchuk, A.V.: Pricing options and variance swaps in Markov-modulated Brownian markets. In: Hidden Markov Models in Finance, pp. 45–68. Springer, Boston (2007)

    Google Scholar 

  19. Latouche, G., Nguyen, G.: Fluid approach to two-sided reflected Markov-modulated Brownian motion. Queue. syst. 80(1-2), 105–125 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  20. Prabhu, N.U., Zhu, Y.: Markov-modulated queueing systems. Queue. Syst. 5(1-3), 215–245 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  21. Miller, H.D.: Absorption probabilities for sums of random variables defined on a finite Markov chain. Proc. Camb. Philos. Soc. 58, 286–298 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  22. Miller, H.D.: A matrix factorization problem in the theory of random variables defined on a finite Markov chain. Proc. Camb. Philos. Soc. 58, 268–285 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  23. Presman, E.L.: Factorization methods, and a boundary value problem for sums of random variables given on a Markov chain. Izv. Akad. Nauk SSSR Ser. Mat. 33, 861–900 (1969)

    MathSciNet  Google Scholar 

  24. Arjas, E., Speed, T.P.: Topics in Markov additive processes. Math. Scand. 33, 171–192 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  25. Prabhu, N.U., Tang, L.C., Zhu, Y.: Some new results for the Markov random walk. J. Math. Phys. Sci. 25(5–6), 635–663 (1991)

    MathSciNet  MATH  Google Scholar 

  26. Newbould, M.: A classification of a random walk defined on a finite Markov chain. Z. Wahrscheinlichkeitstheorie Verw. Gebiete 26, 95–104 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  27. Liu,T.-Y., Ma, W.-Y.: Webpage importance analysis using conditional Markov random walk. In: The 2005 IEEE/WIC/ACM International Conference on Web Intelligence (WI 2005), Compiegne, France, pp. 515–521 (2005)

    Google Scholar 

  28. Kawanabe, M., Binder, A., Müller, C., Wojcikiewicz, W.: Multi-modal visual concept classification of images via Markov random walk over tags. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV), Kona, HI, pp. 396–401 (2011)

    Google Scholar 

  29. Zin, T.T., Tin, P., Toriu, T., Hama, H.: A Markov random walk model for loitering people detection. In: 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, Darmstadt, pp. 680–683 (2010)

    Google Scholar 

  30. Oliver, I.: Markov Processes for Stochastic Modeling. Science&Elsiver Technology Books (2009)

    Google Scholar 

  31. Cinlar, E.: Markov renewal theory. Adv. Appl. Probab. 1, 123–187 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  32. Andronov, A.M., Spiridovska, N.: Markov-modulated linear regression. In: Proceedings’ Book: International Conference on Statistical Models and Methods for Reliability and Survival Analysis and Their Validation (S2MRSA), Bordeaux, France, pp. 24–28 (2012)

    Google Scholar 

  33. Andronov, A.: Markov-modulated multivariate linear regression. Acta et Comment. Univ. Tartuensis de Math. 21(1), 43–50 (2017)

    MathSciNet  MATH  Google Scholar 

  34. Goldfeld, S.M., Richard, E.Q.: A Markov model for switching regressions. J. Econ. 1, 3–16 (1973)

    Article  MATH  Google Scholar 

  35. Cosslett, S.R., Lung-Fei, L.: Serial correlation in discrete variable models. J. Econ. 27, 79–97 (1985)

    Article  Google Scholar 

  36. Chakravarthy, S.R.: The batch Markovian arrival process: a review and future work. Adv. Probab. Theory Stochastic Process. 1, 21–49 (2001)

    Google Scholar 

  37. Naor, P., Yechiali, U.: Queueing problems with heterogenous arrivals and service. Oper. Res. 19, 722–734 (1971)

    Article  MATH  Google Scholar 

  38. Neuts, M.F.: A queue subject to extroneous phase changes. Adv. Appl. Probab. 3, 78–119 (1971)

    Article  MATH  Google Scholar 

  39. Fischer, W., Meier-Hellstern, K.: The Markov-modulated Poisson process (MMPP) cookbook. Perform. Eval. 18(2), 149–171 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  40. Turin, W.: Fitting probabilistic automata via the EM algorithm. Commun. Stat. Stochastic Models 12, 405–424 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  41. Scott, S.L.: Bayesian analysis of a two state Markov modulated Poisson process. J. Comp. Graph. Statist. 8, 662–670 (1999)

    Google Scholar 

  42. Bernardo, J.M., Bayarri, M.J., Berger, J.O., Dawid, A.P., Heckerman, D., Smith, A.F.M. West, M.: The Markov modulated poisson process and Markov poisson cascade with applications to web traffic modeling. Bayesian Stat. (2003)

    Google Scholar 

  43. Prabhu, N.U.: Markov-renewal and Markov-additive processes – a review and some new results. In: Analysis and Geometry 1991 (Korea Adv. Inst. Sci. Tech., Taejŏn), pp. 57–94 (1991)

    Google Scholar 

  44. Zaryadov, I., Korolkova, A., Kulyabov, D., Milovanova, T., Tsurlukov, V.: The survey on markov-modulated arrival processes and their application to the analysis of active queue management algorithms. In: Vishnevskiy, V., Samouylov, K., Kozyrev, D. (eds.) Distributed Computer and Communication Networks. DCCN 2017. Communications in Computer and Information Science, vol. 700. Springer, Cham (2017)

    Google Scholar 

  45. Özekici, S.: Markov modulated Bernoulli process. Math. Methods Oper. Res. 45(3), 311–324 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  46. Özekici, S., Soyer, R.: Bayesian analysis of Markov modulated Bernoulli processes. Math. Methods Oper. Res. 57(1), 125–140 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  47. Andronov, A.M.: Markov-modulated birth-death processes. Autom. Control Comput. Sci. 45(3), 123–132 (2011)

    Article  Google Scholar 

  48. Baum, L.E., Petrie, T.: Statistical inference for probabilistic functions of finite state markov chains. Ann. Math. Stat. 37, 1554–1563 (1966)

    Article  MathSciNet  MATH  Google Scholar 

  49. Ephraim, Y., Merhav, N.: Hidden Markov processes. IEEE Trans. Inf. Theory 48(6), 1518–1569 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  50. Rabiner, L.R., Juang, B.-H.: An introduction to hidden Markov models. IEEE assp magazine 3(1), 4–16 (1986)

    Article  Google Scholar 

  51. Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)

    Article  Google Scholar 

  52. Beal, M.J., Ghahramani, Z., Rasmussen, C.E.: The infinite hidden Markov model. In: Advances in Neural Information Processing Systems (2002)

    Google Scholar 

  53. Elliott, R.J., Aggoun, L., Moore, J.B.: Hidden Markov Models: Estimation and Control, vol. 29. Springer, Heidelberg (2008)

    MATH  Google Scholar 

  54. Starner, T., Pentland, A.: Real-Time American Sign Language Recognition from Video Using Hidden Markov Models. Motion-Based Recognition, pp. 227–243. Springer, Dordrecht (1997)

    Google Scholar 

  55. Cooper, B., Lipsitch, M.: The analysis of hospital infection data using hidden Markov models. Biostatistics 5(2), 223–237 (2004)

    Article  MATH  Google Scholar 

  56. Coast, D.A., et al.: An approach to cardiac arrhythmia analysis using hidden Markov models. IEEE Trans. Biomed. Eng. 37(9), 826–836 (1990)

    Article  Google Scholar 

  57. Zucchini, W., Guttorp, P.: A hidden Markov model for space-time precipitation. Water Resour. Res. 27(8), 1917–1923 (1991)

    Article  Google Scholar 

  58. Mamon, R.S., Elliott, R.J. (eds.): Hidden Markov Models in Finance, vol. 4. Springer, New York (2007)

    MATH  Google Scholar 

  59. Lau, J.K.S., Tham, C.-K.: Hidden Markov models for abnormal event processing in transportation data streams. In: 2012 IEEE 18th International Conference on Parallel and Distributed Systems. IEEE (2012)

    Google Scholar 

  60. Lv, Q., et al.: Big data driven hidden Markov model based individual mobility prediction at points of interest. IEEE Trans. Veh. Technol. 66(6), 5204–5216 (2016)

    Article  Google Scholar 

  61. Tang, B., et al.: Incorporating intelligence in fog computing for big data analysis in smart cities. IEEE Trans. Ind. Inf. 13(5), 2140–2150 (2017)

    Article  Google Scholar 

  62. Manogaran, G., et al.: Machine learning based big data processing framework for cancer diagnosis using hidden Markov model and GM clustering. Wirel. Pers. Commun. 102(3), 2099–2116 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

This work was financially supported by the specific support objective activity 1.1.1.2. “Post-doctoral Research Aid” (Project id. N. 1.1.1.2/16/I/001) of the Republic of Latvia, funded by the European Regional Development Fund.

Nadezda Spiridovska research project No. 1.1.1.2/VIAA/1/16/075 “Non-traditional regression models in transport modelling”.

Author information

Authors and Affiliations

  1. Transport and Telecommunication Institute, Lomonosova 1, Riga, 1019, Latvia

    Nadezda Spiridovska

Authors
  1. Nadezda Spiridovska
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nadezda Spiridovska .

Editor information

Editors and Affiliations

  1. Telematics and Logistics, Transport and Telecommunication Institute, Riga, Latvia

    Igor Kabashkin

  2. Transport and Telecommunication Institute, Riga, Latvia

    Irina Yatskiv

  3. Vilnius Gediminas Technical University, Vilnius, Lithuania

    Olegas Prentkovskis

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Spiridovska, N. (2020). Markov-Modulated Processes, Their Applications and Big Data Cases: State of the Art. In: Kabashkin, I., Yatskiv, I., Prentkovskis, O. (eds) Reliability and Statistics in Transportation and Communication. RelStat 2019. Lecture Notes in Networks and Systems, vol 117. Springer, Cham. https://doi.org/10.1007/978-3-030-44610-9_11

Download citation

Publish with us

Policies and ethics

Search

Navigation