Markov Decision Process and Linear Programming Based Control of MAP/MAP/N Queues

  • András Mészáros
  • Miklós Telek
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8721)


We investigate the control problem of the optimal choice of idle server (if any) for arriving customer in order to minimize the mean system time (waiting time + service time). The considered MAP/MAP/N queue consists of a common infinite buffer and multiple identical servers with MAP service processes whose phases (internal states) are known. Customers arrive according to a MAP (whose phase is also known) and are served with work conserving policy. Idle servers preserve their phases.

We transform the obtained infinite state optimization problem to a finite state one and apply two optimization procedures, policy iteration of finite state MDP and linear programming.


Markov arrival process Markov decision process MAP/ MAP/N queue 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bello, D., Riano, G.: Linear programming solvers for Markov decision processes. In: IEEE Systems and Information Engineering Symposium, pp. 90–95 (2006)Google Scholar
  2. 2.
    Bodrog, L., Gribaudo, M., Horváth, G., Mészáros, A., Telek, M.: Control of queues with map servers: experimental results. In: 8th International Conference on Matrix-Analytic Methods in Stochastic Models (2014)Google Scholar
  3. 3.
    Çinlar, E.: Exceptional paper-Markov renewal theory: A survey. Management Science 21(7), 727–752 (1975)CrossRefzbMATHGoogle Scholar
  4. 4.
    Dantzig, G.B., Thapa, M.N.: Linear programming 2: Theory and Extensions. Springer (2003)Google Scholar
  5. 5.
    Efrosinin, D.: Controlled Queueing Systems with Heterogeneous Servers. PhD thesis, University of Trier (2004)Google Scholar
  6. 6.
    Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer (1997)Google Scholar
  7. 7.
    Gross, D., Miller, D.R.: The randomization technique as a modeling tool and solution procedure for transient Markov processes. Operations Research 32(2), 343–361 (1984)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Latouche, G., Ramaswami, V.: Introduction to Matrix Analytic Methods in Stochastic Modeling. ASA-SIAM Series on Statistics and Applied Probability. Society for Industrial and Applied Mathematics (1999)Google Scholar
  9. 9.
    Littman, M.L., Dean, T.L., Kaelbling, L.P.: On the complexity of solving Markov decision problems. In: Proc. of 11th Conf. on Uncertainty in Artificial Intelligence, pp. 394–402. Morgan Kaufmann Publishers Inc. (1995)Google Scholar
  10. 10.
    Pan, V.: On the complexity of a pivot step of the revised simplex algorithm. Computers & Mathematics with Applications 11(11), 1127–1140 (1985)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Stidham Jr., S., Weber, R.: A survey of Markov decision models for control of networks of queues. Queueing Systems 13(1-3), 291–314 (1993)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • András Mészáros
    • 1
    • 3
  • Miklós Telek
    • 1
    • 2
  1. 1.Budapest University of Technology and EconomicsHungary
  2. 2.MTA-BME Information Systems Research GroupHungary
  3. 3.Inter-University Center for Telecommunications and Informatics DebrecenHungary

Personalised recommendations