Skip to main content
SpringerLink
Log in

Randomization Methods in Optimization and Adaptive Control

  • Conference paper
  • First Online:
Book cover Stochastic Theory and Control

Part of the book series: Lecture Notes in Control and Information Sciences ((LNCIS,volume 280))

Abstract

We consider simultaneous perturbation stochastic approximation (SPSA) methods applied to noise-free problems in optimization and adaptive control. More generally, we consider discrete-time fixed gain stochastic approximation processes that are defined in terms of a random field that is identically zero at some point θ*. The boundedness of the estimator process is enforced by a resetting mechanism. Under appropriate technical conditions the estimator sequence converges to θ* with geometric rate almost surely. This result is in striking contrast to classical stochastic approximation theory where the typical convergence rate is n −1/2. For the proof a discrete-time version of the ODE-method is used and the techniques of [10] are extended. A simple variant of noise free-SPSA is applied to extend a direct controller tuning method named Iterative Feedback Tuning (IFT), see [16]. Using randomization, the number of experiments required to obtain an unbiased estimate of the gradient of the cost function can be reduced significantly for multi-input multi-output systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Benveniste, A., Métivier, M., and Priouret, P. (1990) Adaptive Algorithms and Stochastic Approximations. Springer-Verlag, Berlin.

    MATH  Google Scholar 

  2. Borodin, A. N. (1979) A stochastic approximation procedure in the case of weakly dependent observations, Theory of Probability and Appl. 24, 34–52.

    Article  MathSciNet  Google Scholar 

  3. Chen, H. F., Duncan, T. E. and Pasik-Duncan, B. (1999) A Kiefer-Wolfowitz algorithm with randomized differences, IEEE Trans. Automat. Contr. 44, 442–453.

    Article  MATH  MathSciNet  Google Scholar 

  4. Furstenberg, H. and Kesten, H. (1960) Products of random matrices, Ann. Math. Statist. 31, 457–469.

    Article  MATH  MathSciNet  Google Scholar 

  5. Gerencsér. L. (1989) On a class of mixing processes, Stochastics 26, 165–191.

    MATH  Google Scholar 

  6. Gerencsér. L. (1991) Almost sure exponential stability of random linear differential equations, Stochastics 36, 411–416.

    Google Scholar 

  7. Gerencsér. L. (1991) A representation theorem for the error of recursive estimators. Submitted to SIAM J. Control and Optimization in 1991. Revised in 1997.

    Google Scholar 

  8. Gerencsér. L. (1992) Rate of convergence of recursive estimators, SIAM J. Control and Optimization 30(5), 1200–1227.

    Article  MATH  MathSciNet  Google Scholar 

  9. Gerencsér. L. (1995) Rate of convergence of the LMS algorithm, Systems and Control Letters 24, 385–388.

    Article  MATH  MathSciNet  Google Scholar 

  10. Gerencsér. L. (1996) On fixed gain recursive estimation processes, J. of Mathematical Systems, Estimation and Control 6, 355–358. Retrieval code for full electronic manuscript: 56854.

    MATH  Google Scholar 

  11. Gerencsér. L. (1999) Rate of convergence of moments for a simultaneuous perturbation stochastic approximation method for function minimization, IEEE Trans. Automat. Contr. 44, 894–906.

    Article  MATH  Google Scholar 

  12. Gerencsér. L. (2002) Stability of random iterative mappings. In M. Dror, P. L’ecuyer, and F. Szidarovszky, editors, Modeling Uncertainty. An Examination of its Theory, Methods, and Applications, to appear. Dordrecht, Kluwer, 2002.

    Google Scholar 

  13. Gerencsér, L., Hill, S. D., and Vágó, Zs. (1999) Optimization over discrete sets via SPSA. In Proceedings of the 38-th Conference on Decision and Control, CDC’99, 1791–1794.

    Google Scholar 

  14. Gerencsér, L. and Vágó, Zs. (2001) A stochastic approximation method for noise free optimization. In Proceedings of the European Control Conference, ECC’01, Porto, 1496–1500.

    Google Scholar 

  15. Györfi, L. and Walk, H. (1996) On the average stochastic approximation for linear regression, SIAM J. Control and Optimization 34(1), 31–61.

    Article  MATH  MathSciNet  Google Scholar 

  16. Hjalmarsson, H. (1999) Efficient tuning of linear multivariable controllers using iterative feedback tuning, Int. J. Adapt. Control Signal Process. 13, 553–572.

    Article  MATH  MathSciNet  Google Scholar 

  17. Hjalmarsson, H. (1998) Control of nonlinear systems using Iterative Feedback Tuning. In Proc. 1998 American Control Conference, Philadelphia, 2083–2087.

    Google Scholar 

  18. Hjalmarsson, H., Gevers, M., Gunnarsson, S., and Lequin, O. (1998) Iterative Feedback Tuning: theory and applications. IEEE Control Systems Magazine 18(4), 26–41.

    Article  Google Scholar 

  19. Hjalmarsson, H., Gunnarsson, S., and Gevers, M. (1994) A convergent iterative restricted complexity control design scheme. In Proc. 33rd IEEE CDC, Orlando, Florida, 1735–1740.

    Google Scholar 

  20. Hjalmarsson, H., Gunnarsson, S., and Gevers, M. (1995) Model-free tuning of a robust regulator for a flexible transmission system. European Journal of Control 1, 148–156.

    Google Scholar 

  21. Joslin, J. A. and Heunis, A. J. (2000) Law of the iterated logarithm for a constant-gain linear stochastic gradient algorithm, SIAM J. on Control and Optimization 39, 533–570.

    Article  MATH  MathSciNet  Google Scholar 

  22. Kifer, Y. (1986) Ergodic Theory of Random Transformations. Birkhäuser, 1986.

    Google Scholar 

  23. Kushner, H. J. and Shwartz, A. (1984) Weak convergence and asymptotic properties of adaptive filters with constant gains, IEEE Trans. Informat. Theory 30(2), 177–182.

    Article  MATH  MathSciNet  Google Scholar 

  24. Kushner, H. J. and Yin, G. (1997) Stochastic Approximation Algorithms and Applications. Springer Verlag. New York.

    MATH  Google Scholar 

  25. Ljung, L. and Söderström, T. (1983) Theory and Practice of Recursive Identification. The MIT Press.

    Google Scholar 

  26. Meurers, T. and Veres, S. M. (1999) Iterative design for vibration attenuation. International Journal of Acoustics and Vibration 4(2), 79–83.

    Google Scholar 

  27. Nevelson, M. B. and Khasminskii, R. Z. (1972) Stochastic Approximation and Recursive Estimation. Nauka, Moscow (in Russian).

    Google Scholar 

  28. Solo, V. (1981) The second order properties of a time series recursion, Ann. Stat. 9, 307–317.

    Article  MATH  MathSciNet  Google Scholar 

  29. Spall, J. C. (1992) Multivariate stochastic approximation using a simultaneous perturbation gradient approximation, IEEE Trans. Automat. Contr. 37, 332–341.

    Article  MATH  MathSciNet  Google Scholar 

  30. Spall, J. C. (1997) A one-measurement form of simultaneous perturbation stochastic approximation, Automatica 33, 109–112.

    Article  MATH  MathSciNet  Google Scholar 

  31. Spall, J. C. (2000) Adaptive stochastic approximation by the simultaneous perturbation method, IEEE Trans. Automat. Contr. 45, 1839–1853.

    Article  MATH  MathSciNet  Google Scholar 

  32. Oseledec, V. I. (1968) A multiplicative ergodic theorem. Lyapunov charasteristic numbers for dynamical systems, Trans. Moscow Math. Soc. 19, 197–231.

    MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Hungarian Academy of Sciences, Computer and Automation Institute, 13-17 Kende u., H-1111, Budapest, Hungary

    László Gerencsér & Zsuzsanna Vágó

  2. Dept. of Signals, Sensors and Systems, The Royal Institute of Technology, S-100 44, Stockholm, Sweden

    H. Hjalmarsson

  3. Pázmány Péter Catholic University, Budapest, 28 Szentkirályi u., H-1088, Budapest, Hungary

    Zsuzsanna Vágó

Authors
  1. László Gerencsér
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zsuzsanna Vágó
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. H. Hjalmarsson
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dep. of Mathematics, University of Kansas, 405 Snow Hall, Lawrence, KS, 66045, USA

    Bozenna Pasik-Duncan

Additional information

Dedicated to Tyrone Duncan on occasion of his 60th birthday

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Gerencsér, L., Vágó, Z., Hjalmarsson, H. (2002). Randomization Methods in Optimization and Adaptive Control. In: Pasik-Duncan, B. (eds) Stochastic Theory and Control. Lecture Notes in Control and Information Sciences, vol 280. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48022-6_11

Download citation

  • DOI: https://doi.org/10.1007/3-540-48022-6_11

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-43777-2

  • Online ISBN: 978-3-540-48022-8

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation