Abstract
In this chapter, we review the Finite Difference Stochastic Approximation (FDSA) algorithm, also known as Kiefer-Wolfowitz (K-W) algorithm, and some of its variants for finding a local minimum of an objective function. The K-W scheme is a version of the Robbins-Monro stochastic approximation algorithm and incorporates balanced two-sided estimates of the gradient using two objective function measurements for a scalar parameter. When the parameter is an N-dimensional vector, the number of function measurements using this algorithm scales up to 2N. A one-sided variant of this algorithm in the latter case requires N + 1 function measurements. We present the original K-W scheme, first for the case of a scalar parameter, and subsequently for a vector parameter of arbitrary dimension. Variants including the one-sided version are then presented. We only consider here the case when the objective function is a simple expectation over noisy cost samples and not when it has a long-run average form. The latter form of the cost objective would require multi-timescale stochastic approximation, the general case of which was discussed in Chapter 3. Stochastic algorithms for the long-run average cost objectives will be considered in later chapters.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bhatnagar, S.: Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization. ACM Transactions on Modeling and Computer Simulation 15(1), 74–107 (2005)
Bhatnagar, S.: Adaptive Newton-based smoothed functional algorithms for simulation optimization. ACM Transactions on Modeling and Computer Simulation 18(1), 2:1–2:35 (2007)
Bhatnagar, S., Borkar, V.S.: Multiscale stochastic approximation for parametric optimization of hidden Markov models. Prob. Engg. and Info. Sci. 11, 509–522 (1997)
Bhatnagar, S., Borkar, V.S.: A two time scale stochastic approximation scheme for simulation based parametric optimization. Prob. Engg. and Info. Sci. 12, 519–531 (1998)
Bhatnagar, S., Fu, M.C., Marcus, S.I., Bhatnagar, S.: Two timescale algorithms for simulation optimization of hidden Markov models. IIE Transactions 33(3), 245–258 (2001)
Brandiere, O.: Some pathological traps for stochastic approximation. SIAM J. Contr. and Optim. 36, 1293–1314 (1998)
Cassandras, C.G.: Discrete Event Systems: Modeling and Performance Analysis. Aksen Associates, Boston (1993)
Chong, E.K.P., Ramadge, P.J.: Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times. SIAM J. Cont. and Optim. 31(3), 698–732 (1993)
Chong, E.K.P., Ramadge, P.J.: Stochastic optimization of regenerative systems using infinitesimal perturbation analysis. IEEE Trans. Auto. Cont. 39(7), 1400–1410 (1994)
Fu, M.C.: Convergence of a stochastic approximation algorithm for the GI/G/1 queue using infinitesimal perturbation analysis. J. Optim. Theo. Appl. 65, 149–160 (1990)
Fu, M.C., Hill, S.D.: Optimization of discrete event systems via simultaneous perturbation stochastic approximation. IIE Trans. 29(3), 233–243 (1997)
Ho, Y.C., Cao, X.R.: Perturbation Analysis of Discrete Event Dynamical Systems. Kluwer, Boston (1991)
Kiefer, E., Wolfowitz, J.: Stochastic estimation of the maximum of a regression function. Ann. Math. Statist. 23, 462–466 (1952)
Kleinman, N.L., Spall, J.C., Naiman, D.Q.: Simulation-based optimization with stochastic approximation using common random numbers. Management Science 45, 1570–1578 (1999)
Kushner, H.J., Yin, G.G.: Stochastic Approximation Algorithms and Applications. Springer, New York (1997)
L’Ecuyer, P., Glynn, P.W.: Stochastic optimization by simulation: convergence proofs for the GI/G/1 queue in steady-state. Management Science 40(11), 1562–1578 (1994)
Pemantle, R.: Nonconvergence to unstable points in urn models and stochastic approximations. Annals of Prob. 18, 698–712 (1990)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2013 Springer-Verlag London
About this chapter
Cite this chapter
Bhatnagar, S., Prasad, H., Prashanth, L. (2013). Kiefer-Wolfowitz Algorithm. In: Stochastic Recursive Algorithms for Optimization. Lecture Notes in Control and Information Sciences, vol 434. Springer, London. https://doi.org/10.1007/978-1-4471-4285-0_4
Download citation
DOI: https://doi.org/10.1007/978-1-4471-4285-0_4
Publisher Name: Springer, London
Print ISBN: 978-1-4471-4284-3
Online ISBN: 978-1-4471-4285-0
eBook Packages: EngineeringEngineering (R0)