Locating the minimum of a function when the errors of observation have unknown density

  • Thomas E. Obremski


In considering the problem of locating the point θ at which a functionf achieves its minimum (or maximum) using the Kiefer-Wolfowitz (KW) stochastic approximation procedure, Abdelhamid [1] has shown that if the densityg of the errors obtained in estimating functional values is known, then a transformation of observations leads to methods which under mild conditions have desirable asymptotic properties. We address the more general problem of locating the point of minimum of a function wheng is unknown to the experimenter. In the procedure given in Theorem 4.1 we obtain the same asymptotic results as Abdelhamid in his version of the KW procedure.


Asymptotic Normality Stochastic Approximation Unknown Density Random Variable Versus Measurable Random Variable 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Abdelhamid, S. N. (1973). Transformation of observations in stochastic approximation,Ann. Statist.,1, 1158–1174.MathSciNetCrossRefGoogle Scholar
  2. [2]
    Anbar, D. (1973). On optimal estimation methods using stochastic approximation procedures,Ann. Statist.,1, 1175–1184.MathSciNetCrossRefGoogle Scholar
  3. [3]
    Blum, J. R. (1954). Approximation methods which converge with probability one,Ann. Math. Statist.,25, 737–744.MathSciNetCrossRefGoogle Scholar
  4. [4]
    Dubins, L. E. and Freedman, D. A. (1965). A sharper form of the Borel-Cantelli lemma and the strong law,Ann. Math. Statist.,36, 800–807.MathSciNetCrossRefGoogle Scholar
  5. [5]
    Fabian, V. (1960). Stochastic approximation methods,Czechoslovak Math. J.,10(85), 123–159.MathSciNetzbMATHGoogle Scholar
  6. [6]
    Fabian, V. (1964). A new one-dimensional stochastic approximation method for finding a local minimum of a function,Trans. Third Prague Conf. Information Theory, Statistical Decision Functions, Random Processes, Czechoslovak Academy of Sciences, Prague 85–105.Google Scholar
  7. [7]
    Fabian, V. (1968). On asymptotic normality in stochastic approximation,Ann. Math. Statist.,39, 1327–1332.MathSciNetCrossRefGoogle Scholar
  8. [8]
    Fabian, V. (1971). Stochastic approximation,Optimizing Methods in Statistics (ed. S. S. Rustagi), Academic Press, New York, 439–470.Google Scholar
  9. [9]
    Fabian, V. (1973a) (Referred to in the text as I). Asymptotically efficient stochastic approximation; the RM case,Ann. Statist.,1, 486–495.MathSciNetCrossRefGoogle Scholar
  10. [10]
    Fabian, V. (1973b). Estimation of the derivative of the logarithm of a density,Ann. Statist.,1, 557–561.MathSciNetCrossRefGoogle Scholar
  11. [11]
    Kiefer, J. and Wolfowitz, J. (1952). Stochastic estimation of the maximum of a regression function,Ann. Math. Statist.,23, 462–466.MathSciNetCrossRefGoogle Scholar
  12. [12]
    Loève, Michel (1963).Probability Theory, Third Edition, D. van Nostrand Company, Inc.Google Scholar
  13. [13]
    Robbins, H. and Monro, S. (1951). A stochastic approximation method,Ann. Math. Statist.,22, 400–407.MathSciNetCrossRefGoogle Scholar
  14. [14]
    Venter, J. H. (1967). An extension of the Robbins-Monro procedure,Ann. Math. Statist.,38, 181–190.MathSciNetCrossRefGoogle Scholar

Copyright information

© The Institute of Statistical Mathematics, Tokyo 1982

Authors and Affiliations

  • Thomas E. Obremski
    • 1
  1. 1.The Ohio State UniversityUSA

Personalised recommendations