Skip to main content
SpringerLink
Log in

Perturbation Analysis of Steady-State Performance and Relative Optimization

  • Living reference work entry
  • Latest version View entry history
  • First Online:
Encyclopedia of Systems and Control

  • 24 Accesses

Abstract

We introduce some special theories and methodologies for perturbation analysis (PA) and optimization of steady-state performance and their extensions. Such theories and methodologies utilize the special features of a dynamic systems and usually take different perspectives from the traditional optimization approaches, and therefore they may lead to new insights, new results, and efficient algorithms. The topics discussed include the gradient-based optimization for system with continuous parameters and the direct-comparison-based optimization for systems with discrete policies. Both constitute the relative optimization approach, an alternative to dynamic programming. This approach also applies to continuous-time and continuous-state dynamic systems, leading to a new paradigm of stochastic control.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Bibliography

  • Baxter J, Bartlett PL (2001) Infinite-horizon policy-gradient estimation. J Artif Intell Res 15:319–350

    Article  MathSciNet  Google Scholar 

  • Cao XR (1985) Convergence of parameter sensitivity estimates in a stochastic experiment. IEEE Trans Autom Control 30:834–843

    Article  MathSciNet  Google Scholar 

  • Cao XR (2005) A basic formula for online policy gradient algorithms. IEEE Trans Autom Control 50(5):696–699

    Article  MathSciNet  Google Scholar 

  • Cao XR (2007) Stochastic learning and optimization – a sensitivity-based approach. Springer, New York

    Book  Google Scholar 

  • Cao XR (2015) Optimization of average rewards of time nonhomogeneous Markov chains. IEEE Trans Autom Control 60:1841–1856

    Article  MathSciNet  Google Scholar 

  • Cao XR (2017) Optimality conditions for long-run average rewards with under selectivity and non-smooth features. IEEE Trans Autom Control 62:4318–4332

    Article  Google Scholar 

  • Cao XR (2019) State classification and multi-class optimization of continuous-time and continuous-state Markov processes. IEEE Trans Autom Control 64:3632–3646

    Article  Google Scholar 

  • Cao XR, Wan YW (1998) Algorithms for sensitivity analysis of Markov systems through potentials and perturbation realization. IEEE Trans Control Syst Technol 6:482–494

    Article  Google Scholar 

  • Cao XR, Wan XW (2017) Sensitivity analysis of nonlinear behavior with distorted probability. Math Financ 27:115–150

    Article  MathSciNet  Google Scholar 

  • Cassandras CG, Lafortune S (1999) Introduction to discrete event systems. Kluwer Academic Publishers, Boston

    Book  Google Scholar 

  • Fang HT, Cao XR (2004) Potential-based on-line policy iteration algorithms for Markov decision processes. IEEE Trans Autom Control 49:493–505

    Article  Google Scholar 

  • Fu MC, Hu JQ (1997) Conditional Monte Carlo: gradient estimation and optimization applications. Kluwer Academic Publishers, Boston

    Book  Google Scholar 

  • Glasserman P (1991) Gradient estimation via perturbation analysis. Kluwer Academic Publishers, Boston

    MATH  Google Scholar 

  • Heidelberger P, Cao XR, Zazanis M, Suri R (1988) Convergence properties of infinitesimal perturbation analysis estimates. Manag Sci 34:1281–1302

    Article  MathSciNet  Google Scholar 

  • Ho YC, Cao XR (1983) Perturbation analysis and optimization of queueing networks. J Optim Theory Appl 40:559–582

    Article  MathSciNet  Google Scholar 

  • Ho YC, Cao XR (1991) Perturbation analysis of discrete-event dynamic systems. Kluwer Academic Publisher, Boston

    Book  Google Scholar 

  • Marbach P, Tsitsiklis JN (2001) Simulation-based optimization of Markov reward processes. IEEE Trans Autom Control 46:191–209

    Article  MathSciNet  Google Scholar 

  • Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming. Wiley, New York

    Book  Google Scholar 

  • Robbins H, Monro S (1951) A stochastic approximation method. Ann Math Stat 22:400–407

    Article  MathSciNet  Google Scholar 

  • Xia L, Jia QS, Cao XR (2014) A tutorial on event-based optimization – a new optimization framework. Discret Event Dyn Syst Theory Appl Invited Paper 24:103–132

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This research was supported in part by the Collaborative Research Fund of the Research Grants Council, Hong Kong Special Administrative Region, China, under Grant No. HKUST11/CRF/10 and 610809.

Author information

Authors and Affiliations

  1. Department of Automation, Shanghai Jiao Tong University, Shanghai, China

    Xi-Ren Cao

  2. Institute of Advanced Study, Hong Kong University of Science and Technology, Hong Kong, China

    Xi-Ren Cao

Authors
  1. Xi-Ren Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Electrical and Computer Engineering, Boston University, Boston, MA, USA

    John Baillieul

  2. Automation and Control Solutions, Honeywell, Golden Valley, MN, USA

    Tariq Samad

Section Editor information

  1. Division of Systems Engineering, Center for Information and Systems Engineering, Boston University, Brookline, MA, USA

    Christos G. Cassandras

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer-Verlag London Ltd., part of Springer Nature

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Cao, XR. (2020). Perturbation Analysis of Steady-State Performance and Relative Optimization. In: Baillieul, J., Samad, T. (eds) Encyclopedia of Systems and Control. Springer, London. https://doi.org/10.1007/978-1-4471-5102-9_57-2

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-5102-9_57-2

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-5102-9

  • Online ISBN: 978-1-4471-5102-9

  • eBook Packages: Springer Reference EngineeringReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Chapter history

  1. Latest

    Perturbation Analysis of Steady-State Performance and Relative Optimization
    Published:
    04 January 2020

    DOI: https://doi.org/10.1007/978-1-4471-5102-9_57-2

  2. Original

    Perturbation Analysis of Steady-State Performance and Sensitivity-Based Optimization
    Published:
    11 March 2014

    DOI: https://doi.org/10.1007/978-1-4471-5102-9_57-1

Search

Navigation