Asymptotic Normality of Nonlinear Least Squares under Singular Experimental Designs

  • A. Pázman
  • L. Pronzato
Part of the Springer Optimization and Its Applications book series (SOIA, volume 28)


We study the consistency and asymptotic normality of the LS estimator of a function h(θ) of the parameters θ in a nonlinear regression model with observations \(y_i=\eta(x_i,\theta) +\varepsilon_i\), \(i=1,2\ldots\) and independent errors ε i . Optimum experimental design for the estimation of h(θ) frequently yields singular information matrices, which corresponds to the situation considered here. The difficulties caused by such singular designs are illustrated by a simple example: depending on the true value of the model parameters and on the type of convergence of the sequence of design points \(x_1,x_2\ldots\) to the limiting singular design measure ξ, the convergence of the estimator of h(θ) may be slower than \(1/\sqrt{n}\), and, when convergence is at a rate of \(1/\sqrt{n}\) and the estimator is asymptotically normal, its asymptotic variance may differ from that obtained for the limiting design ξ (which we call irregular asymptotic normality of the estimator). For that reason we focuss our attention on two types of design sequences: those that converge strongly to a discrete measure and those that correspond to sampling randomly from ξ. We then give assumptions on the limiting expectation surface of the model and on the estimated function h which, for the designs considered, are sufficient to ensure the regular asymptotic normality of the LS estimator of h(θ).


Design Point Asymptotic Normality Asymptotic Variance Design Measure Design Sequence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Atkinson, A. Donev, A. (1992). Optimum Experimental Design. Oxford University Press, NY, USA.Google Scholar
  2. Bierens, H. (1994). Topics in Advanced Econometrics. Cambridge University Press, Cambridge.MATHCrossRefGoogle Scholar
  3. Billingsley, P. (1971). Weak Convergence of Measures: Applications in Probability. SIAM, Philadelphia.MATHGoogle Scholar
  4. Elfving, G. (1952). Optimum allocation in linear regression. The Annals of Mathematical Statistics, 23, 255–262.MATHCrossRefMathSciNetGoogle Scholar
  5. Fedorov, V. (1972). Theory of Optimal Experiments. Academic Press, New York.Google Scholar
  6. Gallant, A. (1987). Nonlinear Statistical Models. Wiley, New York.MATHCrossRefGoogle Scholar
  7. Hero, A., Fessler, J., Usman, M. (1996). Exploring estimator bias-variance tradeoffs using the uniform CR bound. IEEE Transactions on Signal Processing, 44, 2026–2041.CrossRefGoogle Scholar
  8. Ivanov, A. (1997). Asymptotic Theory of Nonlinear Regression. Kluwer, Dordrecht.MATHGoogle Scholar
  9. Jennrich, R. (1969). Asymptotic properties of nonlinear least squares estimation. The Annals of Mathematical Statistics, 40, 633–643.MATHCrossRefMathSciNetGoogle Scholar
  10. Kiefer, J. Wolfowitz, J. (1959). Optimum designs in regression problems. The Annals of Mathematical Statistics, 30, 271–294.MATHCrossRefMathSciNetGoogle Scholar
  11. Lehmann, E. Casella, G. (1998). Theory of Point Estimation. Springer, Heidelberg.MATHGoogle Scholar
  12. Pázman, A. (1980). Singular experimental designs. Math. Operationsforsch. Statist., Ser. Statistics, 16, 137–149.Google Scholar
  13. Pázman, A. (1986). Foundations of Optimum Experimental Design. Reidel (Kluwer group), Dordrecht (co-pub. VEDA, Bratislava).Google Scholar
  14. Pázman, A. Pronzato, L. (1992). Nonlinear experimental design based on the distribution of estimators. Journal of Statistical Planning and Inference, 33, 385–402.MATHCrossRefMathSciNetGoogle Scholar
  15. Pázman, A. Pronzato, L. (2006). On the irregular behavior of LS estimators for asymptotically singular designs. Statistics & Probability Letters, 76, 1089–1096.MATHCrossRefMathSciNetGoogle Scholar
  16. Pronzato, L. Pázman, A. (1994). Second-order approximation of the entropy in nonlinear least-squares estimation. Kybernetika, 30, (2)187–198. Erratum. 32(1):104, 1996.MATHMathSciNetGoogle Scholar
  17. Shiryaev, A. (1996). Probability. Springer, Berlin.Google Scholar
  18. Silvey, S. (1980). Optimal Design. Chapman & Hall, London.MATHGoogle Scholar
  19. Sjöberg, J., Zhang, Q., Ljung, L., Benveniste, A., Delyon, B., Glorennec, P.-Y., Hjalmarsson, H., Juditsky, A. (1995). Nonlinear black-box modeling in system identification: a unified overview. Automatica, 31, (12)1691–1724.MATHCrossRefMathSciNetGoogle Scholar
  20. Spivak, M. (1965). Calculus on Manifolds. A Modern Approach to Classical Theorems of Advanced Calculus. W. A. Benjamin, Inc., New York.MATHGoogle Scholar
  21. Stoica, P. (2001). Parameter estimation problems with singular information matrices. IEEE Transactions on Signal Processing, 49, 87–90.CrossRefMathSciNetGoogle Scholar
  22. Wu, C.-F. (1980). Characterizing the consistent directions of least squares estimates. The Annals of Statistics, 8, (4)789–801.MATHCrossRefMathSciNetGoogle Scholar
  23. Wu, C.-F. (1981). Asymptotic theory of nonlinear least squares estimation. The Annals of Statistics, 9, (3)501–513.MATHCrossRefMathSciNetGoogle Scholar
  24. Wu, C.-F. (1983). Further results on the consistent directions of least squares estimators. The Annals of Statistics, 11, (4)1257–1262.MATHMathSciNetGoogle Scholar
  25. Wynn, H. (1972). Results in the theory and construction of D. -optimum experimental designs Journal of the Royal Statistical Society B, 34, 133–147.MATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media LLC 2009

Authors and Affiliations

  1. 1.Department of Applied Mathematics and Statistics, Faculty of Mathematics, Physics and InformaticsComenius UniversityBratislavaSlovakia
  2. 2.Laboratoire I3S, CNRS - UNSA, Les Algorithmes – Bât. Euclide BSophia AntipolisFrance

Personalised recommendations