Markov Decision Process and Linear Programming Based Control of MAP/MAP/N Queues
We investigate the control problem of the optimal choice of idle server (if any) for arriving customer in order to minimize the mean system time (waiting time + service time). The considered MAP/MAP/N queue consists of a common infinite buffer and multiple identical servers with MAP service processes whose phases (internal states) are known. Customers arrive according to a MAP (whose phase is also known) and are served with work conserving policy. Idle servers preserve their phases.
We transform the obtained infinite state optimization problem to a finite state one and apply two optimization procedures, policy iteration of finite state MDP and linear programming.
KeywordsMarkov arrival process Markov decision process MAP/ MAP/N queue
Unable to display preview. Download preview PDF.
- 1.Bello, D., Riano, G.: Linear programming solvers for Markov decision processes. In: IEEE Systems and Information Engineering Symposium, pp. 90–95 (2006)Google Scholar
- 2.Bodrog, L., Gribaudo, M., Horváth, G., Mészáros, A., Telek, M.: Control of queues with map servers: experimental results. In: 8th International Conference on Matrix-Analytic Methods in Stochastic Models (2014)Google Scholar
- 4.Dantzig, G.B., Thapa, M.N.: Linear programming 2: Theory and Extensions. Springer (2003)Google Scholar
- 5.Efrosinin, D.: Controlled Queueing Systems with Heterogeneous Servers. PhD thesis, University of Trier (2004)Google Scholar
- 6.Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer (1997)Google Scholar
- 8.Latouche, G., Ramaswami, V.: Introduction to Matrix Analytic Methods in Stochastic Modeling. ASA-SIAM Series on Statistics and Applied Probability. Society for Industrial and Applied Mathematics (1999)Google Scholar
- 9.Littman, M.L., Dean, T.L., Kaelbling, L.P.: On the complexity of solving Markov decision problems. In: Proc. of 11th Conf. on Uncertainty in Artificial Intelligence, pp. 394–402. Morgan Kaufmann Publishers Inc. (1995)Google Scholar