Abstract
Routing jobs to parallel servers is a common and important task in today’s computer and communication systems. As each routing decision affects the jobs arriving later, determining the (near) optimal decisions is non-trivial. In this paper, we apply reinforcement learning techniques to the job routing problem with heterogeneous servers and a general cost structure. We study the convergence of the reinforcement learning to a near-optimal policy (that we can determine by other means), and compare its performance against heuristic policies such as Join-the-Shortest-Queue (JSQ) and Shortest-Expected-Delay (SED).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
RND (random) chooses the server independently in random using some probabilities \(p_k\), JSQ chooses the queue with the least number of jobs, and SED the queue with the shortest expected response time, i.e., the admission cost to queue i is \((n_i+1)/\mu _i\).
References
Haight, F.A.: Two queues in parallel. Biometrika 45(3–4), 401–410 (1958)
Watkins, C.: Learning from delayed rewards, Ph.D. dissertation, Cambridge University (1989)
Hyytiä, E., Righter, R., Samúelsson, S.G.: Beyond the shortest queue routing with heterogeneous servers and general cost function. In: ValueTools, December 2017
Whittle, P.: Optimal Control: Basics and Beyond. Wiley, New York (1996)
Hyytiä, E.: Lookahead actions in dispatching to parallel queues. Perform. Eval. 70(10), 859–872 (2013). (IFIP Performance 2013)
Hyytiä, E., Virtamo, J.: Dynamic routing and wavelength assignment using first policy iteration. In: The Fifth IEEE ISCC 2000, pp. 146–151, July 2000
Howard, R.A.: Dynamic Probabilistic Systems, Volume II: Semi-Markov and Decision Processes. Wiley Interscience, New York (1971)
Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957)
Acknowledgements
This work was supported by the Academy of Finland in the FQ4BD project (grant no. 296206) and by the University of Iceland Research Fund in the RL-STAR project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Samúelsson, S.G., Hyytiä, E. (2018). Applying Reinforcement Learning to Basic Routing Problem. In: Takahashi, Y., Phung-Duc, T., Wittevrongel, S., Yue, W. (eds) Queueing Theory and Network Applications. QTNA 2018. Lecture Notes in Computer Science(), vol 10932. Springer, Cham. https://doi.org/10.1007/978-3-319-93736-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-93736-6_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93735-9
Online ISBN: 978-3-319-93736-6
eBook Packages: Computer ScienceComputer Science (R0)