Optimization of Stochastic Discrete Event Dynamic Systems: A Survey of Some Recent Results

  • Alexei A. Gaivoronski
Conference paper
Part of the Lecture Notes in Economics and Mathematical Systems book series (LNE, volume 374)


Models of discrete event dynamic systems (DEDS) include finite state machines [24], Petri nets [35], finitely recursive processes [26], communicating sequential processes [18], queuing models [41] among others. They become increasingly popular due to important applications in manufacturing systems, communication networks, computer systems. We would consider here a system which evolution or sample path consists of the sequence
$$y\left( {x,\omega} \right) = \left\{ {\left( {{t_0},{z_0}} \right),\left( {{t_1},{z_1}} \right),...,\left( {{t_s},{z_s}} \right)} \right\},{t_i} = {t_i}\left( {x,\omega} \right),{z_i} = {z_i}\left( {x,\omega} \right)$$
where zi(x,ω)) ∈ W is the state of the system during the time interval \({t_i}\left( {x,\omega} \right) \le t < {t_{i + 1}}\left( {x,\omega} \right)\), x∈X⊆Rn is the set of control parameters and ω∈Ω is an element of some probability space (Ω, 𝔹, ℙ). Particular rules which govern transitions between states at time moments ti. can be specified in the framework of one of the models mentioned above. For describing the time behavior the generalized semi- Markov processes proved to be useful [47].


Discrete Event Sample Path Perturbation Analysis Evolution Step Discrete Event System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    F. Archetti, A. Gaivoronski and A. Sciomachen, Sensitivity analysis and optimization of stochastic Petri Nets, Preprint, University of Milano, 1990.Google Scholar
  2. 2.
    V. M. Aleksandrov, V. I. Sysoyev and V. V. Shemeneva, Stochastic optimization, Eng. Cybern., v. 5, 1968, p. 11–16.Google Scholar
  3. 3.
    X. R. Cao, Convergence of parameter sensitivity estimates in a stochastic experiment, IEEE Transactions on Automatic Control, v. AC-30, No. 9, 1985, p. 845–853.Google Scholar
  4. 4.
    M. A. Crane and D. L. Iglehart. Simulating stable stochastic systems. III. Regenerative processes and discrete-event simulations, Oper. Res. vol.23, 1975, p.33–45.CrossRefGoogle Scholar
  5. 5.
    Yu. Ermoliev, Methods of Stochastic Programming, Nauka, Moscow, 1976 (in Russian).Google Scholar
  6. 6.
    Yu. Ermoliev and A. A. Gaivoronski, Stochastic programming techniques for optimization of discrete event systems, Annals of Operations Research, 1991.Google Scholar
  7. 7.
    Yu. Ermoliev, Optimization of discrete event systems described by semi-Markov processes, Lecture at the conference “Computationally intensive methods in Simulation and Optimization”, Vienna, August 1990.Google Scholar
  8. 8.
    Yu. Ermoliev and Wets, R. J.-B. eds. Numerical Techniques for Stochastic Optimization, Springer-Verlag, Berlin, 1988.Google Scholar
  9. 9.
    A.A. Gaivoronski, Approximation methods of solution of stochastic programming problems,- Kibernetika, 2, 1982 (in Russian, English translation in: Cybernetics, v. 18, No.2).Google Scholar
  10. 10.
    A.A. Gaivoronski, Interactive Program SQG-PC for Solving Stochastic Programming Problems on IBM PC/XT/AT Compatibles. User Guide. Working Paper WP-88–11, IIASA, Laxenburg, 1988.Google Scholar
  11. 11.
    A.A. Gaivoronski, Augmented Perturbation Analysis for optimization of discrete event systems, Preprint, Institute of Cybernetics, Kiev, 1990.Google Scholar
  12. 12.
    P. Glasserman and W. B. Gong. Smoothed perturbation analysis for a class of discreet-event systems. IEEE Trans. on Automatic Control, 35(11): 1218–1230, 1990.CrossRefGoogle Scholar
  13. 13.
    P.W. Glynn, Optimization of stochastic systems, in: Proceedings of 1986 Winter Simulation Conference, 1986.Google Scholar
  14. 14.
    P.W. Glynn and J.L. Sanders, Monte Carlo Optimization of Stochastic Systems: Two New Approaches. Proc. 1986 ASME Computing in Engineering Conference, (Chicago, IL) 1986.Google Scholar
  15. 15.
    W.B. Gong and Y. C. Ho, Smoothed (Conditional) Perturbation Analysis of discrete event dynamic systems, IEEE Transactions on Automatic Control, AC-32, 1987, 856–866.Google Scholar
  16. 16.
    W. B. Gong, C. G. Cassandras and J. Pan, Perturbation analysis of a multiclass queueing system with admission control. IEEE Transactions on Automatic Control, vol.36, No. 6, 1991, p. 707–723.CrossRefGoogle Scholar
  17. 17.
    P. Heidelberger, Xi-Ren Cao, M. A. Zazanis and R. Suri. Convergence properties of Infinitesimal Perturbation Analysis estimates, Management Science, v. 34, No. 11, 1988.Google Scholar
  18. 18.
    C. A. R. Hoare, Communicating Sequential Processes. Englewood Cliffs, NJ: Prentice-Hall International, 1985.Google Scholar
  19. 19.
    Y. C. Ho, Performance evaluation and perturbation analysis of discrete event dynamic systems. IEEE Transactions on Automatic Control, vol. AC-32, No. 7, 1987, p. 563–572.Google Scholar
  20. 20.
    Y. C. Ho and S. Li, Extensions of Infinitesimal Perturbation Analysis, IEEE Transactions on Automatic Control, AC-33, 1988, p. 427–438.CrossRefGoogle Scholar
  21. 21.
    Y.C. Ho, M. A. Eyler and T. T. Chien. A gradient technique for general buffer storage design in a serial production line. Int. J. Prod. Res., v. 17, No.6, 1979, p. 557–580.CrossRefGoogle Scholar
  22. 22.
    Y.C. Ho, (ed.). A selected and annotated bibliography on perturbation analysis, Lecture Notes in Control and Information Sciences, Vol.103, Springer-Verlag, 1987, pp.162–178.Google Scholar
  23. 23.
    Y.C. Ho, L. Shi, L. Dai and W. Gong. Optimizing discrete event dynamic systems via the gradient surface method, Manuscript, Harward University, 1990.Google Scholar
  24. 24.
    J. E. Hopcroft and J.D. Ullman, Introduction to Automata Theory, Languages and Computation, Reading, MA: Addison-Wesley, 1979.Google Scholar
  25. 25.
    J. Q. Hu and Y.-C. Ho. An infinitesimal perturbation analysis algorithm for a multiclass G/G/1 queue. OR Letters, 9:35–44, 1990.Google Scholar
  26. 26.
    K. Inan and P. Varaiya, “Finitely recursive process models for discrete event systems”, IEEE Transactions on Automatic Control, v. 33, No.7, 1988, p. 626–639.CrossRefGoogle Scholar
  27. 27.
    S.H. Jacobson and Schruben L.W., Techniques for simulation response optimization, Operations Research Letters, Feb. 1989, 1–9.Google Scholar
  28. 28.
    P. Kall, Stochastic Linear Programming, Springer Verlag, Berlin, 1976.CrossRefGoogle Scholar
  29. 29.
    J. Kiefer and J. Wolfowitz. Stochastic estimation of a maximum of a regression function, Ann. Math. Statist. 23, 1952, 462–466.CrossRefGoogle Scholar
  30. 30.
    H. Kushner, and D.S. Clark. Stochastic Approximation for Constrained and Unconstrained Systems, Appl. Math. 26, Springer, 1978.Google Scholar
  31. 31.
    K. Marti, Lecture at the Conference “Computationally intensive methods in Simulation and Optimization”, IIASA, Laxenburg, 1990.Google Scholar
  32. 32.
    D.L. McLeich and S. Rollans. Conditioning for variance reduction in estimating the sensitivity of simulations. To be published in Annals of Operations Research, 1992.Google Scholar
  33. 33.
    M. S. Meketon, “Optimization in simulation: A survey of recent results”, Proceedings of the 1987 Winter Simulation Conference, A. Thesen, H. Grant, W. David Keiton (eds), pp. 58–61.Google Scholar
  34. 34.
    R.H. Myers, Response Surface Methodology, Allyn and Bacon, Boston, Massachusetts, 1987.Google Scholar
  35. 35.
    J.L. Peterson, Petri Net Theory and the Modelling of Systems. Englewood Cliffs, NJ: Prentice Hall, 1981.Google Scholar
  36. 36.
    G. Ch. Pflug, On line optimization of simulated Markovian processes. Mathematics of OR, 1990.Google Scholar
  37. 37.
    G. Ch. Pflug, Derivatives of probability measures — concepts and applications to the optimization of stochastic systems, in: Discrete Events Systems: Models and Applications. IIASA Conference, Sopron, Hungary, August 3–7, 1987, P. Varaiya and A. B. Kurzhanski (eds.), Lecture Notes in Control and Information Sciences, Springer Verlag, 1988, p. 162–178.Google Scholar
  38. 38.
    G.Ch. Pflug, Optimization of simulated discrete event processes, Preprint TR-ISI/Stamcom 87, University of Vienna, 1990.Google Scholar
  39. 39.
    M.I. Reiman and A. Weiss, Sensitivity analysis via likelihood ratios, in: Proceedings of the 1986 Winter Simulation Conference, 1986, p. 285–289.Google Scholar
  40. 40.
    R.Y. Rubinstein, The score function approach of sensitivity analysis of computer simulation models. Math. and Computation in Simulation, vol.28, 1986, p. 351–379.CrossRefGoogle Scholar
  41. 41.
    R.Y. Rubinstein, Monte Carlo Optimization, Simulation and Sensitivity Analysis of Queuing Networks. New York, Wiley, 1986.Google Scholar
  42. 42.
    R.Y. Rubinstein. How to optimize discrete-event systems from a single sample path by the score function method. Annals of Operations Research 27 (1991) 175–212.CrossRefGoogle Scholar
  43. 43.
    R. Suri, Infinitesimal Perturbation Analysis of General Discrete Event Systems, J. Assoc. Comput. Mach., 34, 1987, 686–717CrossRefGoogle Scholar
  44. 44.
    R. Suri, Perturbation Analysis: The State of the Art and Research Issues Explained via the GI/G/1 Queue, Proceedings of the IEEE, v. 77, No. 1, 1989, 114–137.CrossRefGoogle Scholar
  45. 45.
    R. Suri and Y.T. Leung, Single run optimization of a SIMAN model for automatic assembly systems, Proceedings of the 1987 Winter Simulation Conference, A.Thesen, H.Grant, W. D. Keiton (eds.)Google Scholar
  46. 46.
    R. J.-B. Wets, Stochastic programming: solution techniques and approximation schemes, in: Mathematical Programming: the State of the Art, 1982, eds. A. Bachern, M. Grotschel and B. Korte, Springer Verlag, 1983, p. 566–603.CrossRefGoogle Scholar
  47. 47.
    W. Whitt, Continuity of generalized semi-Markov process, Math. Oper. Research, vol.5, 1980, p. 494–501.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1992

Authors and Affiliations

  • Alexei A. Gaivoronski
    • 1
  1. 1.V.Glushkov Institute of CyberneticsKievUSSR

Personalised recommendations