On-Line Optimization of Queues Using Infinitesimal Perturbation Analysis

  • Edwin K. P. Chong
Part of the The IMA Volumes in Mathematics and its Applications book series (IMA, volume 73)


Infinitesimal perturbation analysis (IPA) is a method for estimating the gradient of a performance measure in a discrete event system by observing a single sample path of the system. The method lends itself naturally to recursive optimization using gradient-based algorithms. Such algorithms can be used in on-line optimization applications, or in single-run optimization of simulation models. We describe the use of such algorithms for optimization of single server queues. We give sufficient conditions that guarantee convergence of the algorithm to the optimizing point. The convergence proof is simple, and provides insight into how the algorithm behaves under different update times.

Key words

Queues on-line optimization infinitesimal perturbation analysis stochastic approximation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    S. Asmussen, Applied Probability and Queues. Chichester: John Wiley & Sons, 1987.zbMATHGoogle Scholar
  2. [2]
    J. D. Bartusek and A. M. Makowski, “On stochastic approximations driven by sample averages: Convergence results via the ODE method,” manuscript, Electrical Engineering Department and Institute for Systems Research, University of Maryland, College Park, MD 20742, 1994.Google Scholar
  3. [3]
    X. R. Cao, “Realization probability in closed Jackson queueing networks and its applications,” Adv. in Appl. Prob., vol. 19, pp. 708– 738, 1987.Google Scholar
  4. [4]
    E. K. P. Chong, “On distributed stochastic optimization of regenerative systems using IPA,” in Proc. 31st IEEE Conf. on Decision and Control, pp. 3203–3208, Dec. 1992.Google Scholar
  5. [5]
    E. K. P. Chong, “Optimization of tandem networks using a distributed asynchronous algorithm with IPA estimators,” in Proc. 1992 American Control Conference, pp. 3196–3200, June 1992.Google Scholar
  6. [6]
    E. K. P. Chong find P. J. Ramadge, “Convergence of recursive optimization algorithms using infinitesimal perturbation analysis estimates,” Discrete Event Dynamic Systems: Theory and Applications, vol. 1, no. 4, pp. 339–372, June 1992.Google Scholar
  7. [7]
    E. K. P. Chong and P. J. Ramadge, “On a stochastic optimization algorithm using IPA which updates after every customer,” in Proc. 28th Allerton Conf. on Communication, Control, and Computing, pp. 658–667, Oct. 1990.Google Scholar
  8. [8]
    E. K. P. Chong and P. J. Ramadge, “On regenerative stochastic approximations with application to optimization using IPA,” in Proc. 25th Annual Conf. on Information Sciences and Systems, pp. 743–748, Mar. 1991.Google Scholar
  9. [9]
    E. K. P. Chong and P. J. Ramadge, “Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times,” SIAM J. Control and Optimization, vol. 31, no. 3, pp. 698–732, May 1993.MathSciNetzbMATHCrossRefGoogle Scholar
  10. [10]
    E. K. P. Chong and P. J. Ramadge, “Stochastic optimization of regenerative systems using infinitesimal perturbation analysis,” IEEE Trans. Automat. Control, July 1994, to appear.Google Scholar
  11. [11]
    M. C. Fu, “Convergence of a stochastic approximation algorithm for the GI/G/1 queue using infinitesimal perturbation analysis,” J. Opt. Th. Appl., vol. 65, no. 1, pp. 149–160, 1990.zbMATHCrossRefGoogle Scholar
  12. [12]
    P. Glasserman, Gradient Estimation via Perturbation Analysis. Norwell, Massachusetts: Kluwer Academic Publishers, 1991.Google Scholar
  13. [13]
    P. Glasserman, “Regenerative derivatives of regenerative sequences,” Adv. in Appl. Prob., vol. 25, pp. 116–139, 1993.MathSciNetzbMATHCrossRefGoogle Scholar
  14. [14]
    P. Glasserman, J. Q. Hu, and S. G. Strickland, “Strongly consistent steady-state derivative estimates,” Probability in the Engineering and Informational Sciences, vol. 5, pp. 391–413, 1991.MathSciNetzbMATHCrossRefGoogle Scholar
  15. [15]
    A. Gut, Stopped Random Walks: Limit Theorems and Applications. New York: Springer-Verlag, 1988.zbMATHGoogle Scholar
  16. [16]
    P. Heidelberger, X.-R. Cao, M. A. Zazanis, and R. Suri, “Convergence properties of infinitesimal perturbation analysis estimates,” Management Sci., vol. 34, no. 11, pp. 1281–1301, Nov. 1988MathSciNetzbMATHCrossRefGoogle Scholar
  17. [17]
    Y.-C. Ho and X.-R. Cao, Perturbation Analysis of Discrete Event Dynamic Systems. Norwell, Massachusetts: Kluwer Academic Publishers, 1991.Google Scholar
  18. [18]
    J. Q. Hu, “Convexity of sample path performance and strong consistency of infinitesimal perturbation analysis,” IEEE Trans. Automat. Contr., vol. AC-37, no. 2, pp. 258–262, Feb. 1992.Google Scholar
  19. [19]
    R. L. Kashyap, C. C. Blaydon, and K. S. Fu, “Stochastic approximation,” in A Prelude to Neural Networks: Adaptive and Learning Systems (J. M. Mendel, ed.), Englewood Cliffs, New Jersey: PTR Prentice Hall, pp. 329–355, 1994.Google Scholar
  20. [20]
    L. Kleinrock, Queueing Systems, Volume I: Theory. New York: J. Wiley, 1975.Google Scholar
  21. [21]
    P. Konstantopoulos and M. Zazanis, ‘Sensitivity analysis for stationary and ergodic queues,’ Adv. in Appl. Prob., vol. 24, pp. 738–750, 1992.MathSciNetzbMATHCrossRefGoogle Scholar
  22. [22]
    H. J. Kushner and D. S. Clark, Stochastic Approximation Methods for Constrained and Unconstrained Systems, vol. 26 of Appl. Math. Sci. New York: Springer, 1978.Google Scholar
  23. [23]
    P. L’Ecuyer and P. Glynn, “Stochastic optimization by simulation: Convergence proofs for the GI/G/1 queue in steady-state,” Management Science, to appear.Google Scholar
  24. [24]
    L. Ljung, G. Pflug, and H. Walk, Stochastic Approximation and Optimization of Random Systems. Basel: Birkhauser Verlag, 1992.zbMATHCrossRefGoogle Scholar
  25. [25]
    M. S. Meketon, “Optimization in simulation: A survey of recent results,” in Proc. 1987 Winter Simulation Conference, pp. 58–67, 1987.Google Scholar
  26. [26]
    R. Suri and Y. T. Leung, “Single run optimization of discrete event simulations — An empirical study using the M/M/l queue,” IIE Transactions, vol. 21, no. 1, pp. 35–49, Mar. 1989.CrossRefGoogle Scholar
  27. [27]
    R. Suri and M. A. Zazanis, “Perturbation analysis gives strongly consistent sensitivity estimates for the M/G/l queue,” Management Sci., vol. 34, no. 1, pp. 39–64, Jan. 1988MathSciNetzbMATHCrossRefGoogle Scholar
  28. [28]
    Q.-Y. Tang and H.-F. Chen, “Convergence of perturbation analysis based optimization algorithm with fixed-length observation period,” manuscript, Institute of Systems Science, Academia Sinica, Beijing, 1993.Google Scholar
  29. [29]
    F. J. Vázquez-Abad, “Stochastic recursive algorithms for optimal routing in queueing networks,” Ph.D. Thesis, Division of Applied Mathematics, Brown University, May 1989.Google Scholar
  30. [30]
    F. J. Vázquez-Abad and H. J. Kushner, “Stochastic approximation methods for systems of interest over an infinite horizon,” Tech. Rep. LCDS # 94–4, Lefschetz Center for Dynamical Systems and Center for Control Sciences, Brown University, Providence, RI 02912, Jan. 1994.Google Scholar
  31. [31]
    Y. Wardi, “Simulation-based stochastic algorithms for optimizing GI/G/1 queues,” preprint, Dept. of Industrial Engr., Ben Gurion University of the Negev, 1988.Google Scholar
  32. [32]
    Y. Wardi and J. Q. Hu, “Strong consistency of infinitesimal perturbation analysis for tandem queueing networks,” J. Discrete Event Dynamic Systems, vol. 1, no. 1, pp. 37–59, May 1991.zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag New York, Inc. 1995

Authors and Affiliations

  • Edwin K. P. Chong
    • 1
  1. 1.School of Electrical EngineeringPurdue UniversityWest LafayetteUSA

Personalised recommendations