Policy Optimization by Neural Network and Its Application to Queueing Allocation Problem

  • Hisashi Sato
  • Yutaka Matsumoto
  • Norio Okino
Conference paper


The problem of allocating an arriving customer to one of parallel servers has been actively studied in queueing theory for load balancing in computer networks or in multi-processor systems. To theoretically derive the optimal allocation policy, the assumption of identical servers is usually required. However this assumption is unrealistic in many applications. This paper considers the queueing allocation problem with non-identical servers and multi-class customers. The goal is to optimize the allocation policy with respect to the mean delay of an arbitrary customer. To this end, we represent the allocation policy by a neural network; namely, we allocate an arriving customer according to the output of the neural network, where the inputs to the neural-net are the numbers of queueing customers at each server and the class of the arrival. By using the simulated annealing method, we search the optimal allocation policy in the weight space. Numerical results show that the present procedure significantly reduces the mean delay in comparison to an empirical policy.


Service Time Allocation Policy Service Time Distribution Simulated Annealing Method Identical Server 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    J. Walrand, An introduction to queueing networks, Prentice Hall, NJ, 1988.MATHGoogle Scholar
  2. [2]
    M. Ohnishi, “On two control problems of queues,” Journal of the Operations Research Society of Japan, no. 4, pp. 179–184, 1991 (in Japanese).Google Scholar
  3. [3]
    W. Winston, “Optimality of the shortest line discipline,” J. Appl. Prob., vol. 14, pp. 181–189, 1977.MathSciNetMATHCrossRefGoogle Scholar
  4. [4]
    R.R. Weber, “On the optimal assignment of customers to parallel servers,” J. Appl. Prob., vol. 15, pp. 406–413, 1978.MATHCrossRefGoogle Scholar
  5. [5]
    J. Takinami, Y. Matsumoto, N. Okino, “Performance evaluation of neural networks applied to queueing allocation problem,” Artificial Neural Nets and Genetic Algorithms, R.F. Albrecht et. al. (eds.), Springer-Verlag, pp. 316–323, 1993.Google Scholar
  6. [6]
    Y. Matsumoto, “On optimization of polling policy represented by neural network,” Computer Communication Review, vol. 24, no. 4, pp. 181–190, October 1994.CrossRefGoogle Scholar
  7. [7]
    T. Kohonen, “The self-organizing map,” Proceedings of the IEEE, vol. 78, no. 9, pp. 1464–1480, September 1990.CrossRefGoogle Scholar
  8. [8]
    S. Markon, H. Kita and Y. Nishikawa, “Reinforcement learning for stochastic system control by using a feature extraction with BP neural networks,” Tech. Rep. of IEICE, NC91-126, pp. 209–214, 1991.Google Scholar
  9. [9]
    E. Aarts and J. Korst, Simulated annealing and Boltzmann machines, John Wiley & Sons, 1989.MATHGoogle Scholar

Copyright information

© Springer-Verlag/Wien 1995

Authors and Affiliations

  • Hisashi Sato
    • 1
  • Yutaka Matsumoto
    • 2
  • Norio Okino
    • 1
  1. 1.Division of Applied Systems Science Faculty of EngineeringKyoto UniversityJapan
  2. 2.I.T.S., Inc.Japan

Personalised recommendations