Introduction to Discrete Event Systems pp 617-740 | Cite as

# Sensitivity Analysis and Concurrent Estimation

## Abstract

After going through all the previous chapters, it would be natural for readers to conclude that DES are inherently complex and hard to analyze, regardless of the modeling framework adopted. It would also be natural to wonder whether there are any properties at all in DES that we can exploit in our effort to develop mathematical techniques for analysis, design, and control.

In Chap.1, we saw that systems with event-driven dynamics cannot be modeled through differential (or difference) equations, leaving us with inadequate mathematical tools to analyze their behavior. In Chaps. 2–5, we developed a number of models, both timed and untimed. Although these models certainly enhance our understanding of DES and give us solid foundations for analysis, one cannot help but notice that they involve rather elaborate notation and sometimes intricate definitions. In fact, the complexity of these models was such that in order to proceed with any manageable analysis of stochastic DES we were forced, in Chaps. 7–9, to work with the special class of Markov chains. Even within this class, simple formulae in closed form are hard to come by; in Chap. 8, we saw that outside some simple queueing systems at steady state, analysis becomes extremely complicated. This complexity has led to discrete-event simulation as the only universal tool at this point for studying DES, especially stochastic ones. However, as we saw in Chap. 10, simulation has its own limitations. First, it is based on statistical experiments whose validity must always be checked; we saw, for example, that the problem of selecting the length of simulation runs in seeking estimates of steady-state performance measures is far from simple. Second, even with a new generation of very fast computers, simulation remains a slow and costly approach, certainly not suited for real-time applications. A typical problem one faces in the design or control of a system is that of determining how performance is affected as a function of some parameter θ. In the absence of analytical expressions of the form *J*(θ), where *J*(θ) is a performance measure, one is forced to simulate the system repeatedly, at least once for each value of θ of interest. It is easy to see how tedious a process this becomes for complex systems where θ is in fact a vector of parameters or perhaps a list of alternative designs or control policies to be explored.

## Keywords

Service Time Sample Path Busy Period Sample Function Idle Period## Preview

Unable to display preview. Download preview PDF.

## Selected References

## ∎ Perturbation Analysis

- — Bao, G., CG. Cassandras, and M. Zazanis, “First and Second Derivative Estimators for Closed Jackson-Like Queueing Networks Using Perturbation Analysis Techniques,”
*Journal of Discrete Event Dynamic Systems*, Vol. 7, No. 1, pp. 29–67, 1997.MATHCrossRefGoogle Scholar - — Brémaud, P., and F.J.Vázquez-Abad, “On the Pathwise Computation of Derivatives with Respect to the Rate of a Point Process: The Phantom RPA Method,”
*Queueing Systems*, Vol. 10, pp. 249–270, 1992.MATHCrossRefGoogle Scholar - — Cao, X., “Convergence of Parameter Sensitivity Estimates in a Sto-chastic Experiment,”
*IEEE Transactions on Automatic Control*, Vol. 30, pp. 834–843, 1985.CrossRefGoogle Scholar - — Cao, X.,
*Realization Probabilities: The Dynamics of Queueing Systems*, Springer-Verlag, 1994.Google Scholar - — Cassandras, C.G., W.B. Gong, and J.I. Lee, “On the Robustness of Perturbation Analysis Estimators for Queueing Systems with Unknown Distributions,”
*Journal of Optimization Theory and Applications*,Vol. 30, No. 3, pp. 491–519, 1991.CrossRefMathSciNetGoogle Scholar - — Cassandras, C.G., and V. Julka, “Scheduling Policies Using Mar-ked/Phantom Slot Algorithms,”
*Queueing Systems*, Vol. 20, pp. 207–254, 1995.MATHCrossRefMathSciNetGoogle Scholar - — Dai, L., and Y.C. Ho, “Structural Infinitesimal Perturbation Analysis (SIPA) for Derivative Estimation of Discrete Event Dynamic Systems,”
*IEEE Transactions on Automatic Control*, Vol. 40, pp. 1154–1166, 1995.MATHCrossRefMathSciNetGoogle Scholar - — Fu, M.C., and J.Q. Hu, “Smoothed Perturbation Analysis for Queues with Finite Buffers,”
*Queueing Systems*, Vol. 14, pp. 57–78, 1993.MATHCrossRefMathSciNetGoogle Scholar - — Fu, M.C., and J.Q. Hu, “Smoothed Perturbation Analysis Derivative Estimation for Markov Chains,”
*Operations Research Letters*, Vol. 15, pp. 241–251, 1994.MATHCrossRefMathSciNetGoogle Scholar - — Fu, M.C., and J.Q. Hu,
*Conditional Monte Carlo: Gradient Estimation and Optimization Applications*, Kluwer Academic Publishers, Boston, 1997.MATHGoogle Scholar - — Glasserman, P., and W.B. Gong, “Smoothed Perturbation Analysis for a Class of Discrete Event Systems,”
*IEEE Transactions on Automatic Control*, Vol. AC-35, No. 11, pp. 1218–1230, 1990.CrossRefMathSciNetGoogle Scholar - — Glasserman, P.,
*Gradient Estimation via Perturbation Analysis*, Kluwer Academic Publishers, Boston, 1991.MATHGoogle Scholar - — Gong, W.B., and Y.C. Ho, “Smoothed Perturbation Analysis of Discrete Event Systems,”
*IEEE Transactions on Automatic Control*, Vol. AC-32, No. 10, pp. 858–866, 1987.CrossRefMathSciNetGoogle Scholar - — Gong, W.B., and P. Glasserman, “Perturbation Analysis of the
*M/G/1/K*Queue,”*Proceedings of 27th IEEE Conference on Decision and Control*, 1988.Google Scholar - — Gong, W.B., C.G. Cassandras, and J.Pan, “Perturbation Analysis of a Multiclass Queueing System with Admission Control,”
*IEEE Transactions on Automatic Control*, Vol. AC-36, No. 6, pp. 707—723, 1991.CrossRefMathSciNetGoogle Scholar - — Gong, W.B., and H. Schulzrinne, “Perturbation Analysis of the
*M/G/1/K*Queue,”*Mathematics and Computers in Simulation*, Vol. 34, pp. 1—19, 1992.CrossRefGoogle Scholar - — Heidelberger, P., X. Cao, M. Zazanis, and R. Suri, “Convergence Properties of Infinitesimal Perturbation Analysis Estimates,”
*Management Science*, Vol. 34, No. 11, pp. 1281—1302, 1988.MATHCrossRefMathSciNetGoogle Scholar - — Ho, Y.C., and X. Cao, “Perturbation Analysis and Optimization of Queueing Networks,”
*Journal of Optimization Theory and Applications*, Vol. 40, No. 4, pp. 559—582, 1983.MATHCrossRefMathSciNetGoogle Scholar - — Ho, Y.C., and X. Cao,
*Perturbation Analysis of Discrete Event Dynamic Systems*, Kluwer Academic Publishers, Boston, 1991.MATHGoogle Scholar - — Ho, Y.C., and C.G. Cassandras, “A New Approach to the Analysis of Discrete Event Dynamic Systems,”
*Automatica*, Vol. 19, No. 2, pp. 149—167, 1983.MATHCrossRefMathSciNetGoogle Scholar - — Ho, Y.C., and C.G. Cassandras, “Perturbation Analysis for Control and Optimization of Queueing Systems: An Overview and the State of the Art,” in
*Frontiers in Queueing*(J. Dshalalow, Ed.), CRC Press, pp. 395—420, 1997.Google Scholar - — Ho, Y.C., and J.Q. Hu, “An Infinitesimal Perturbation Analysis Algorithm for a Multiclass
*G/G/*1 Queue,”*Operations Research Letters*, Vol. 9, pp. 35—44, 1990.MATHCrossRefMathSciNetGoogle Scholar - — Ho, Y.C., and S. Li, “Extensions of Perturbation Analysis of Discrete Event Dynamic Systems,”
*IEEE Transactions on Automatic Control*, Vol. AC-33, pp. 427—438, 1988.CrossRefMathSciNetGoogle Scholar - — Ho, Y.C., X. Cao, and C.G. Cassandras, “Infinitesimal and Finite Perturbation Analysis for Queueing Networks,”
*Automatica*, Vol. 19, pp. 439—445, 1983.MATHCrossRefGoogle Scholar - — Ho, Y.C., M.A. Eyler, and D.T. Chien, “A Gradient Technique for General Buffer Storage Design in a Serial Production Line,”
*Intl. Journal of Production Research*, Vol. 17, pp. 557—580, 1979.CrossRefGoogle Scholar - — Hu, J.Q., “Convexity of Sample Path Performances and Strong Consistency of Infinitesimal Perturbation Analysis Estimates,”.Google Scholar
- — Hu, J.Q., and S.G. Strickland, “Strong Consistency of Sample Path Derivative Estimates,”
*Applied Mathematics Letters*, Vol. 3, No. 4, pp. 55—58, 1990. IEEE Transactions on Automatic Control, Vol. AC-37, No. 2, pp. 258—262, 1992.MATHCrossRefMathSciNetGoogle Scholar - — Suri, R., “Implementation of Sensitivity Calculations on a Monte-Carlo Experiment,”
*Journal of Optimization Theory and Applications*, Vol. 40, No. 4, pp. 625—630, 1983.MATHCrossRefMathSciNetGoogle Scholar - — Suri, R., “Perturbation Analysis: The State of the Art and Research Issues Explained via the
*GI/G/*1 Queue,”*Proceedings of the IEEE*, Vol. 77, No. 1, pp. 114—137, 1989.CrossRefGoogle Scholar - — Suri, R., and X. Cao, “The Phantom and Marked Customer Methods for Optimization of Closed Queueing Networks with Blocking and General Service Times,”
*ACM Performance Evaluation Review*, Vol. 12, No. 3, pp. 234—256, 1986.Google Scholar - — Suri, R., and M. Zazanis, “Perturbation Analysis Gives Strongly Consistent Sensitivity Estimates for the
*M/G*/1 Queue,”*Management Science*, Vol. 34, pp. 39–64, 1988.MATHCrossRefMathSciNetGoogle Scholar - — Vakili, P., and Y.C. Ho, “Infinitesimal Perturbation Analysis of a Multiclass Routing Problem,”
*Proceedings of 25th Allerton Conference on Communication, Control, and Computing*, pp. 279–286, 1987.Google Scholar - — Wardi, Y., W.B. Gong, C.G. Cassandras, and M.H. Kallmes, “Smoo-thed Perturbation Analysis for a Class of Piecewise Constant Sample Performance Functions,”
*Journal of Discrete Event Dynamic Systems*, Vol. 1, No. 4, pp. 393–414, 1992.MATHGoogle Scholar - — Wardi, Y., and J.Q. Hu, “Strong Consistency of Infinitesimal Perturbation Analysis for Tandem Queueing Networks,”
*Journal of Discrete Event Dynamic Systems*, Vol. 1, No. 1, pp. 37–60, 1991.MATHCrossRefGoogle Scholar - — Zazanis, M., and R. Suri, “Perturbation Analysis of the
*GI/G*/1 Queue,” Technical Report, IE/MS Dept., Northwestern University, 1989.Google Scholar

## ∎ Applications of Sensitivity Estimation to Optimization

- — Cassandras, C.G., and J.I. Lee, “Applications of Perturbation Techniques to Optimal Resource Sharing in Discrete Event Systems,”
*Proceedings of 1988 American Control Conference*, pp. 450–455, 1988.Google Scholar - — Cassandras, C.G., M.V. Abidi, and D. Towsley “Distributed Routing with On-Line Marginal Delay Estimation,”
*IEEE Transactions on Communications*, Vol. COM-38, No. 3, pp. 348–359, 1990.CrossRefGoogle Scholar - — Chong, E.K.P., and P.J. Ramadge, Convergence of Recursive Optimization Algorithms Using Infinitesimal Perturbation Analysis Estimates,
*Journal of Discrete Event Dynamic Systems*, vol. 1, No. 4, pp. 339–372, 1992.MATHGoogle Scholar - — Fu, M.C., “Convergence of a Stochastic Approximation Algorithm for the
*GI/G*/1 Queue Using Infinitesimal Perturbation Analysis,”*Journal of Optimization Theory and Applications*, Vol. 65, pp. 149–160, 1990.MATHCrossRefMathSciNetGoogle Scholar - — Suri, R., and Y.T. Leung, “Single Run Optimization of Discrete Event Simulations - An Empirical Study using the
*M/M*/1 Queue,”*IIE Transactions*, Vol. 21, No. 1, pp. 35–49, 1989.CrossRefGoogle Scholar - — Vázquez-Abad, F.J., C.G. Cassandras, and V. Julka, “Centralized and Decentralized Asynchronous Optimization of Stochastic Discrete Event Systems,”
*IEEE Transactions on Automatic Control*, Vol. AC-43, No. 5, pp. 631–655, 1998.CrossRefGoogle Scholar

## ∎ Likelihood Ratio Methodology for Sensitivity Estimation

- — Aleksandrov, V.M., V.J. Sysoyev, and V.V. Shemeneva, “Stochastic Optimization,”
*Engineering Cybernetics*, Vol. 5, pp. 11–16, 1968.Google Scholar - — Glynn, P.W., “Likelihood Ratio Gradient Estimation: An Overview,”
*Proceedings of 1987 Winter Simulation Conference*, pp. 366–375, 1987.Google Scholar - — Reiman, M.I., and A. Weiss, “Sensitivity Analysis via Likelihood Ratio,”
*Operations Research*, Vol. 37, pp. 830–844, 1989.MATHCrossRefMathSciNetGoogle Scholar - — Rubinstein, R.Y., and A. Shapiro,
*Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method*, Wiley, New York, 1993.MATHGoogle Scholar

## ∎ Perturbation Analysis for Stochastic Flow Models

- — Cassandras, C.G., Y. Wardi, B. Melamed, G. Sun, and C. G. Panayiotou, “Perturbation Analysis for On-Line Control and Optimization of Stochastic Fluid Models,”
*IEEE Transactions on Automatic Control*, Vol. AC-47, pp. 1234–1248, 2002.CrossRefMathSciNetGoogle Scholar - — Sun, G., C.G. Cassandras, Y. Wardi, C.G. Panayiotou, and G. Riley, “Perturbation Analysis and Optimization of Stochastic Flow Networks,”
*IEEE Transactions on Automatic Control*, Vol. AC-49, pp. 2113–2128, 2004.MathSciNetGoogle Scholar - — Sun, G., C.G. Cassandras, and C.G. Panayiotou, “Perturbation Analysis of Mul-ticlass Stochastic Fluid Models,”
*Journal of Discrete Event Dynamic Systems*, Vol. 14, pp. 267–307, 2004.MATHCrossRefMathSciNetGoogle Scholar - — Wardi, Y., and B. Melamed, “Variational Bounds and Sensitivity Analysis of Traffic Processes in Continuous Flow Models,”
*Journal of Discrete Event Dynamic Systems*, Vol. 11, pp. 249–282, 2002.CrossRefMathSciNetGoogle Scholar - — Wardi, Y., B. Melamed, C.G. Cassandras, and C.G. Panayiotou, “IPA Gradient Estimators in Single-Node Stochastic Fluid Models,”
*Journal of Optimization Theory and Applications*, Vol. 115, pp. 369–406, 2002.MATHCrossRefMathSciNetGoogle Scholar - — Yan, A., and W. Gong, “Fluid Simulation for High-Speed Networks with Flow-Based Routing,”
*IEEE Transactions on Information Theory*, Vol. 45, pp. 1588–1599, 1999.MATHCrossRefMathSciNetGoogle Scholar - — Yu, H., and C.G. Cassandras, “Perturbation Analysis for Production Control and Optimization of Manufacturing Systems,”
*Automatica*, Vol. 40, pp. 945–956, 2004.MATHCrossRefMathSciNetGoogle Scholar - — Yu, H., and C.G. Cassandras, “Perturbation Analysis of Feedback-Controlled Stochastic Flow Systems,”
*IEEE Transactions on Automatic Control*, Vol. AC-49, pp. 1317–1332, 2004.CrossRefMathSciNetGoogle Scholar - — Yu, H., and C.G. Cassandras, “A New Paradigm for On-Line Management of Communication Networks with Multiplicative Feedback Control,” in
*Performance Evaluation and Planning Methods for the Next Generation Internet*, A. Girard, B. Sansó, and F. Vázquez-Abad, (Eds.), Springer-Verlag, New York, 2005.Google Scholar - — Yu, H., and C.G. Cassandras, “Perturbation Analysis and Feedback Control of Communication Networks Using Stochastic Hybrid Models,”
*Journal of Nonlinear Analysis*, Vol. 65, pp. 1251–1280, 2006.MATHCrossRefMathSciNetGoogle Scholar

## ∎ Concurrent Estimation

- — Cassandras, C.G. (Ed.), Special Issue on Parallel Simulation and Optimization of Discrete Event Dynamic Systems,
*Journal of Discrete Event Dynamic Systems*, Vol. 5, No. 2/3, 1995.Google Scholar - — Cassandras, C.G., and C.G. Panayiotou, “Concurrent Sample Path Analysis of Discrete Event Systems,”
*Journal of Discrete Event Dynamic Systems*, 1999.Google Scholar - — Cassandras, CG., and S.G. Strickland, “Sample Path Properties of Timed Discrete Event Systems,”
*Proceedings of the IEEE*, Vol. 77, No. 1, pp. 59–71, 1989a.CrossRefMathSciNetGoogle Scholar - — Cassandras, C.G., and S.G. Strickland, “On-Line Sensitivity Analysis of Markov Chains,”
*IEEE Transactions on Automatic Control*, Vol. AC-34, No. 1, pp. 76–86, 1989b.CrossRefMathSciNetGoogle Scholar - — Cassandras, C.G., and S.G. Strickland, “Observable Augmented Systems for Sensitivity Analysis of Markov and Semi-Markov Processes,”
*IEEE Transactions on Automatic Control*, Vol. AC-34, No. 10, pp. 1026–1037, 1989c.CrossRefMathSciNetGoogle Scholar - — Cassandras, C.G., J.I. Lee, and Y.C. Ho, “Efficient Parametric Analysis of Performance Measures for Communication Networks,”
*IEEE Journal on Selected Areas in Communications*, Vol. 8, No. 9, pp. 1709–1722, 1990.CrossRefGoogle Scholar - — Vakili, P., “A Standard Clock Technique for Efficient Simulation,”
*Operations Research Letters*, Vol. 10, pp. 445–452, 1991.MATHCrossRefMathSciNetGoogle Scholar

## ∎ Miscellaneous

- — Bratley, P.B., B.L. Fox,, and L.E. Schrage,
*A Guide to Simulation*, 2nd Edition, Springer-Verlag, New York, 1987.Google Scholar - — Cassandras, CG., L. Dai, and C.G. Panayiotou, “Ordinal Optimization for Deterministic and Stochastic Resource Allocation,”
*IEEE Transactions on Automatic Control*, Vol. AC-43, No. 7, pp. 881–900, 1998.CrossRefMathSciNetGoogle Scholar - — Ho, Y.C, R. Sreenivas, and P. Vakili, “Ordinal Optimization of Discrete Event Dynamic Systems,”
*Journal of Discrete Event Dynamic Systems*, Vol. 2, No. 1, pp. 61–88, 1992.MATHCrossRefGoogle Scholar - — Gallager, R.G., “A Minimum Delay Routing Algorithm Using Distributed Computation,”
*IEEE Transactions on Communications*, Vol. COM-23, pp. 73–85, 1977.CrossRefMathSciNetGoogle Scholar - — Rubinstein, R.V.,
*Monte Carlo Optimization, Simulation, and Sensitivity Anal-ysis of Queueing Networks*, Wiley, New York, 1986.Google Scholar