Markovian Modeling of Real Data Traffic: Heuristic Phase Type and MAP Fitting of Heavy Tailed and Fractal Like Samples

  • András Horváth
  • Miklós Telek
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2459)


In order to support the effective use of telecommunication infrastructure, the “random” behavior of traffic sources has been studied since the early days of telephony. Strange new features, like fractal like behavior and heavy tailed distributions were observed in high speed packet switched data networks in the early ’90s. Since that time a fertile research aims to find proper models to describe these strange traffic features and to establish a robust method to design, dimension and operate such networks.

In this paper we give an overview of methods that, on the one hand, allow us to capture important traffic properties like slow decay rate, Hurst parameter, scaling factor, etc., and, on the other hand, makes possible the quantitative analysis of the studied systems using the effective analysis approach called matrix geometric method.

The presentation of this analysis approach is associated with a discussion on the properties and limits of Markovian fitting of the typical non- Markovian behavior present in telecommunication networks.


Partition Function Arrival Process Fractional Brownian Motion Interarrival Time Haar Wavelet 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    A. T. Andersen and B. F. Nielsen. A markovian approach for modeling packet traffic with long-range dependence. IEEE Journal on Selected Areas in Communications, 16(5):719–732, 1998.CrossRefGoogle Scholar
  2. 2.
    S. Asmussen and O. Nerman. Fitting Phase-type distributions via the EM algorithm. In Proceedings: “Symposium i Advent Statistik”, pages 335–346, Copenhagen, 1991.Google Scholar
  3. 3.
    J. Beran. Statistics for long-memory processes. Chapman and Hall, New York, 1994.zbMATHGoogle Scholar
  4. 4.
    A. W. Berger. On the index of dispersion for counts for user demand modeling. In ITU, Madrid, Spain, June 1994. Study Group 2, Question 17/2.Google Scholar
  5. 5.
    A. Bobbio, A. Horváth, M. Scarpa, and M. Telek. Acyclic discrete phase type distributions: Properties and a parameter estimation algorithm. submitted to Performance Evaluation, 2000.Google Scholar
  6. 6.
    S. C. Borst, O. J. Boxma, and R. Nunez-Queija. Heavy tails: The effect of the service discipline. In Tools 2002, pages 1–30, London, England, April 2002. Springer, LNCS 2324.Google Scholar
  7. 7.
    G. E. P. Box, G. M Jenkins, and C. Reinsel. Time Series Analysis: Forecasting and Control. Prentice Hall, Englewood Cliff, N.J., third edition, 1994.zbMATHGoogle Scholar
  8. 8.
    E. Castillo. Extreme Value Theory in Engineering. Academic Press, San Diego, California, 1988.zbMATHGoogle Scholar
  9. 9.
    M. E. Crovella and M. S. Taqqu. Estimating the heavy tail index from scaling properties. Methodology and Computing in Applied Probability, 1(1):55–79, 1999.zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    A. Cumani. On the canonical representation of homogeneous Markov processes modelling failure-time distributions. Microelectronics and Reliability, 22:583–602, 1982.CrossRefGoogle Scholar
  11. 11.
    R. El Abdouni Khayari, R. Sadre, and B. Haverkort. Fitting world-wide web request traces with the EM-algorithm. In Proc. of SPIE, volume 4523, pages 211–220, Denver, USA, 2001.Google Scholar
  12. 12.
    R. El Abdouni Khayari, R. Sadre, and B. Haverkort. A valiadation of the pseudo self-similar traffic model. In Proc. of IPDS, Washington D.C., USA, 2002.Google Scholar
  13. 13.
    A. Feldman, A. C. Gilbert, and W. Willinger. Data networks as cascades: Investigating the multifractal nature of internet WAN traffic. Computer communication review, 28/4:42–55, 1998.CrossRefGoogle Scholar
  14. 14.
    A. Feldman and W. Whitt. Fitting mixtures of exponentials to long-tail distributions to analyze network performance models. Performance Evaluation, 31:245–279, 1998.CrossRefGoogle Scholar
  15. 15.
    W. Fischer and K. Meier-Hellstern. The Markov-modulated Poisson process (MMPP) cookbook. Performance Evaluation, 18:149–171, 1992.CrossRefMathSciNetGoogle Scholar
  16. 16.
    H. J. Fowler and W. E. Leland. Local area network traffic characteristics, with implications for broadband network congestion management. IEEE JSAC, 9(7):1139–1149, 1991.Google Scholar
  17. 17.
    R. Fox and M. S. Taqqu. Large sample properties of parameter estimates for strongly dependent stationary time series. The Annals of Statistics, 14:517–532, 1986.zbMATHMathSciNetCrossRefGoogle Scholar
  18. 18.
    C. W. J. Granger and R. Joyeux. An introduction to long-memory time series and fractional differencing. Journal of Time Series Analysis, 1:15–30, 1980.zbMATHMathSciNetCrossRefGoogle Scholar
  19. 19.
    H. Heffes and D. M. Lucantoni. A Markov-modulated characterization of packetized voice and data traffic and related statistical multiplexer performance. IEEE Journal on Selected Areas in Communications, 4(6):856–868, 1986.CrossRefGoogle Scholar
  20. 20.
    B. M. Hill. A simple general approach to inference about the tail of a distribution. The Annals of Statistics, 3:1163–1174, 1975.zbMATHMathSciNetCrossRefGoogle Scholar
  21. 21.
    A. Horváth, G. I. Rózsa, and M. Telek. A map fitting method to approximate real traffic behaviour. In 8th IFIP Workshop on Performance Modelling and Evaluation of ATM & IP Networks, pages 321–12, Ilkley, England, July 2000.Google Scholar
  22. 22.
    A. Horváth and M. Telek. Approximating heavy tailed behavior with phase type distributions. In 3rd International Conference on Matrix-Analytic Methods in Stochastic models, Leuven, Belgium, 2000.Google Scholar
  23. 23.
    A Horváth and M. Telek. A markovian point process exhibiting multifractal behaviour and its application to traffic modeling. In Proc. of MAM4, Adelaide, Australia, 2002.Google Scholar
  24. 24.
    A. Horváth and M. Telek. Phfit: A general phase-type fitting tool. In Proc. of 12th Performance TOOLS, volume 2324 of Lecture Notes in Computer Science, pages 82–91, Imperial College, London, April 2002.Google Scholar
  25. 25.
    M. Kratz and S. Resnick. The qq-estimator and heavy tails. Stochastic Models, 12:699–724, 1996.zbMATHMathSciNetCrossRefGoogle Scholar
  26. 26.
    A. Lang and J. L. Arthur. Parameter approximation for phase-type distributions. In S. R. Chakravarty and A. S. Alfa, editors, Matrix-analytic methods in stochastic models, Lecture notes in pure and applied mathematics, pages 151–206. Marcel Dekker, Inc., 1996.Google Scholar
  27. 27.
    G. Latouche and V. Ramaswami. Introduction to Matrix-Analytic Methods in Stochastic Modeling. Series on statistics and applied probability. ASA-SIAM, 1999.Google Scholar
  28. 28.
    W. E. Leland, M. Taqqu, W. Willinger, and D. V. Wilson. On the self-similar nature of ethernet traffic (extended version). IEEE/ACM Transactions in Networking, 2:1–15, 1994.CrossRefGoogle Scholar
  29. 29.
    D. M. Lucantoni. New results on the single server queue with a batch Markovian arrival process. Commun. Statist.-Stochastic Models, 7(1):1–46, 1991.zbMATHMathSciNetCrossRefGoogle Scholar
  30. 30.
    B. B. Mandelbrot and J. W. Van Ness. Fractional Brownian motions, fractional noises and applications. SIAM Review, 10:422–437, 1969.CrossRefGoogle Scholar
  31. 31.
    B. B. Mandelbrot and M. S. Taqqu. Robust R/S analysis of long-run serial correlation. In Proceedings of the 42nd Session of the International Statistical Institute, volume 48, Book 2, pages 69–104, Manila, 1979. Bulletin of the I.S.I.Google Scholar
  32. 32.
    K.S. Meier. A fitting algorithm for Markov-modulated Poisson processes having two arrival rates. European Journal of Operations Research, 29:370–377, 1987.zbMATHCrossRefGoogle Scholar
  33. 33.
    M. Neuts. Probability distributions of phase type. In Liber Amicorum Prof. Emeritus H. Florin, pages 173–206. University of Louvain, 1975.Google Scholar
  34. 34.
    M.F. Neuts. Matrix Geometric Solutions in Stochastic Models. Johns Hopkins University Press, Baltimore, 1981.zbMATHGoogle Scholar
  35. 35.
    M.F. Neuts. Structured stochastic matrices of M/G/1 type and their applications. Marcel Dekker, 1989.Google Scholar
  36. 36.
    I. Norros. A storage model with self-similar imput. Queueing Systems, 16:387–396, 1994.zbMATHCrossRefMathSciNetGoogle Scholar
  37. 37.
    I. Norros. On the use of fractional brownian motion in the theorem of connectionless networks. IEEE Journal on Selected Areas in Communications, 13:953–962, 1995.CrossRefGoogle Scholar
  38. 38.
    A. Ost and B. Haverkort. Modeling and evaluation of pseudo self-similar traffic with infinite-state stochastic petri nets. In Proc. of the Workshop on Formal Methods in Telecommunications, pages 120–136, Zaragoza, Spain, 1999.Google Scholar
  39. 39.
    S. Resnick. Heavy tail modeling and teletraffic data. The Annals of Statistics, 25:1805–1869, 1997.zbMATHCrossRefMathSciNetGoogle Scholar
  40. 40.
    S. Resnick and C. Starica. Smoothing the hill estimator. Advances in Applied Probability, 29:271–293, 1997.zbMATHCrossRefMathSciNetGoogle Scholar
  41. 41.
    J. Rice. Mathematical Statistics and Data Analysis. Brooks/Cole Publishing, Pacific Grove, California, 1988.zbMATHGoogle Scholar
  42. 42.
    R. H. Riedi. An introduction to multifractals. Technical report, Rice University, 1997. Available at
  43. 43.
    R. H. Riedi, M. S. Crouse, V. J. Ribeiro, and R. G. Baraniuk. A multifractal wavelet model with application to network traffic. IEEE Transactions on Information Theory, 45:992–1018, April 1999.Google Scholar
  44. 44.
    S. Robert and J.-Y. Le Boudec. New models for pseudo self-similar traffic. Performance Evaluation, 30:1997, 57–68.CrossRefGoogle Scholar
  45. 45.
    M. Roughan, D. Veitch, and M. Rumsewicz. Numerical inversion of probability generating functions of power-law tail report, 1997.Google Scholar
  46. 46.
    T. Rydén. Parameter estimation for Markov Modulated Poisson Processes. Stochastic Models, 10(4):795–829, 1994.zbMATHMathSciNetCrossRefGoogle Scholar
  47. 47.
    T. Ryden. An EM algorithm for estimation in Markov modulated Poisson processes. Computational statist. and data analysis, 21:431–447, 1996.zbMATHCrossRefMathSciNetGoogle Scholar
  48. 48.
    T. Rydén. Estimating the order of continuous phase-type distributions and markovmodulated poisson processes. Stochastic Models, 13:417–433, 1997.zbMATHCrossRefGoogle Scholar
  49. 49.
    B. Ryu and S. B. Lowen. Point process models for self-similar network traffic, with applications. Stochastic models, 14, 1998.Google Scholar
  50. 50.
    M. Telek and A. Heindl. Moment bounds for acyclic discrete and continuous phase-type distributions of second order. In in proc. of Eighteenth Annual UK Performance Engineering Workshop (UKPEW), Glasgow, UK, 2002.Google Scholar
  51. 51.
    J. Lévy Véhel and R. H. Riedi. Fractional brownian motion and data traffic modeling: The other end of the spectrum. In C. Tricot J. Lévy Véhel, E. Lutton, editor, Fractals in Engineering, pages 185–202. Springer, 1997.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • András Horváth
    • 1
  • Miklós Telek
    • 1
  1. 1.Dept. of TelecommunicationsBudapest University of Technology and EconomicsUK

Personalised recommendations