Skip to main content
SpringerLink
Log in

Local Optimality Criteria Based on Asymptotic Normality

  • Chapter
  • First Online:
Book cover Design of Experiments in Nonlinear Models

Part of the book series: Lecture Notes in Statistics ((LNS,volume 212))

  • 2647 Accesses

Abstract

The approach considered in this chapter is probably the most common for designing experiments in nonlinear situations. It consists in optimizing a scalar function of the asymptotic covariance matrix of the estimator and thus relies on asymptotic normality, as considered in Chaps. 3 and 4. Design based on more accurate characterizations of the precision of the estimation will be considered in Chap. 6. Additionally to asymptotic normality, the approach also supposes that the asymptotic covariance matrix takes the form of the inverse of an information matrix, as it is the case, for instance, for weighted LS with optimum weights, see Sect. 3.1.3, or maximum likelihood estimation, see Sect. 4.2.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Another interpretation is that locally optimum design is based on the Cramér–Rao bound and the Fisher information matrix; see Sect. 4.4.2. Note, however, that the Cramér–Rao inequality gives a lower bound, whereas an upper bound would be more suitable, but unfortunately is not available.

  2. 2.

    When necessary, we shall extend the definition of a concave criterion Φ( ⋅) over the whole set \(\mathbb{M}\) of symmetric p ×p matrices by allowing Φ( ⋅) to take the value − ∞. All the criteria considered are proper functions; that is, they satisfy Φ(M) < ∞ for all \(\mathbf{M} \in \mathbb{M}\), and their effective domain \(\mathrm{dom}(\Phi ) =\{ \mathbf{M} \in \mathbb{M} : \Phi (\mathbf{M}) > -\infty \}\) is non empty.

  3. 3.

    The reparameterization may also be nonlinear, with \(\mathbf{A} = \partial \beta /{\partial \theta }^{\top }\vert _{{\theta }^{0}}\), where θ 0 denotes the nominal value for θ used in locally optimum design.

  4. 4.

    Up to a change of sign, since Kiefer considered criteria that should be minimized.

  5. 5.

    From Cauchy–Schwarz inequality, ρ F (M) ≥ p with equality if and only if M is proportional to the identity matrix.

  6. 6.

    For any \(\mathbf{M}_{1},\,\mathbf{M}_{2} \in \mathbb{M}\), M 1 ≽ M 2 (M 1 ≻ M 2) if and only if \(\mathbf{M}_{1} -\mathbf{M}_{2} \in {\mathbb{M}}^{\geq }\) (\({\mathbb{M}}^{>}\)).

  7. 7.

    For any \(\mathbf{M}_{1},\,\mathbf{M}_{2} \in \mathbb{M}\), M 1 is better that M 2 according to Schur’s ordering if and only if \(\Phi _{E_{k}}(\mathbf{M}_{1}) \geq \Phi _{E_{k}}(\mathbf{M}_{2})\) for k = 1, …, p, see (5.12), with strict inequality for one k at least.

  8. 8.

    It should be stressed here that neither positive homogeneity nor concavity do necessarily matter for the optimization of a differentiable criterion. Positive homogeneity is important when we wish to compare criteria through their efficiency. Equivalence with a concave criterion is crucial for preventing the existence of local maxima, but concavity itself is required in some particular situations only (typically, when the criterion is not differentiable). One may refer, e.g., to Avriel [2003, Chap. 6], for extensions of the notion of convexity.

  9. 9.

    Notice that it implies that log[ − Φ c ( ⋅)] is convex on \(\mathbb{M}_{\mathbf{c}}^{\geq }\), which is a stronger results than Φ c ( ⋅) being concave on \(\mathbb{M}_{\mathbf{c}}^{\geq }\).

  10. 10.

    Note that log[Φ q, I  + (M)] is therefore concave on \({\mathbb{M}}^{>}\) for q ∈ ( − 1, ∞), so that log[trace(M  − q)]1 ∕ q is convex, which is stronger than the convexity results mentioned, e.g., in [Kiefer, 1974] for [trace(M  − q)]1 ∕ q.

  11. 11.

    The now classical denomination “equivalence theorem” can be considered as a tribute to the work of Kiefer and Wolfowitz [1960]; we use this term throughout the book although “Necessary and Sufficient condition for optimality” would be appropriate too.

  12. 12.

    ξ = ξ 1 ⊗ ξ 2 is the joint design measure on \(\mathcal{X} = \mathcal{X}_{1} \times \mathcal{X}_{2}\) with ξ(dx, dy) given by the product ξ 1(dx) ×ξ 2(dy) of the marginal measures, respectively, on \(\mathcal{X}_{1}\) and \(\mathcal{X}_{2}\).

  13. 13.

    We have m = p(p + 1) ∕ 2 when the design criterion Φ( ⋅) is strictly isotonic since an optimum design matrix M  ∗  (unique if Φ( ⋅) is strictly concave or is equivalent to a strictly concave criterion) necessarily lies on the boundary of \(\mathcal{M}_{\theta }(\Xi )\). Indeed, when M is in the interior of \(\mathcal{M}_{\theta }(\Xi )\), there exists α > 1 such that \(\alpha \mathbf{M} \in \mathcal{M}_{\theta }(\Xi )\) and Φ(α M) > Φ(M). From Caratheodory’s theorem, M  ∗  can then be written as the linear combination of p(p + 1) ∕ 2 elements of \(\mathcal{M}_{\theta }(\mathcal{X})\).

  14. 14.

    Possible developments may involve the use of bounds, obtained, e.g., from Kantorovich-type inequalities; see Bloomfield and Watson [1975], Pečarić et al. [1996], and Liu and Neudecker [1997].

References

  • Alexéev, V., E. Galéev, and V. Tikhomirov (1987). Recueil de Problèmes d’Optimisation. Moscou: MIR.

    Google Scholar 

  • Atwood, C. (1969). Optimal and efficient designs of experiments. Ann. Math. Statist. 40(5), 1570–1602.

    Article  MathSciNet  MATH  Google Scholar 

  • Atwood, C. (1980). Convergent design sequences for sufficiently regular optimality criteria, II singular case. Ann. Statist. 8(4), 894–912.

    Article  MathSciNet  MATH  Google Scholar 

  • Avriel, M. (2003). Nonlinear Programming. Analysis and Methods. New York: Dover. [Originally published by Prentice Hall, 1976].

    Google Scholar 

  • Bloomfield, P. and G. Watson (1975). The inefficiency of least squares. Biometrika 62(1), 121–128.

    Article  MathSciNet  MATH  Google Scholar 

  • Box, G. and H. Lucas (1959). Design of experiments in nonlinear situations. Biometrika 46, 77–90.

    MathSciNet  MATH  Google Scholar 

  • Chaloner, K. and I. Verdinelli (1995). Bayesian experimental design: a review. Statist. Sci. 10(3), 273–304.

    Article  MathSciNet  MATH  Google Scholar 

  • Cook, D. and V. Fedorov (1995). Constrained optimization of experimental design (invited discussion paper). Statistics 26, 129–178.

    Article  MathSciNet  MATH  Google Scholar 

  • Cook, D. and W. Wong (1994). On the equivalence between constrained and compound optimal designs. J. Amer. Statist. Assoc. 89(426), 687–692.

    Article  MathSciNet  MATH  Google Scholar 

  • Cook, R., D. Hawkins, and S. Weisberg (1993). Exact iterative computation of the robust multivariate minimum volume ellipsoid estimator. Statist. Probab. Lett. 16, 213–218.

    Article  MathSciNet  Google Scholar 

  • Cramér, H. (1946). Mathematical Methods of Statistics. Princeton, N.J.: Princeton University Press.

    MATH  Google Scholar 

  • Davies, L. (1992). The asymptotics of Rousseeuw’s minimum volume ellipsoid estimator. Ann. Statist. 20(4), 1828–1843.

    Article  MathSciNet  MATH  Google Scholar 

  • de la Garza, A. (1954). Spacing of information in polynomial regression. Ann. Math. Statist. 25, 123–130.

    Article  MATH  Google Scholar 

  • Dem’yanov, V. and V. Malozemov (1974). Introduction to Minimax. New York: Dover.

    Google Scholar 

  • Dette, H. (1993). Elfving’s theorem for D-optimality. Ann. Statist. 21(2), 753–766.

    Article  MathSciNet  MATH  Google Scholar 

  • Dette, H. and V. Melas (2011). A note on de la Garza phenomenon for locally optimal designs. Ann. Statist. 39(2), 1266–1281.

    Article  MathSciNet  MATH  Google Scholar 

  • Dette, H. and W. Studden (1993). Geometry of E-optimality. Ann. Statist. 21(1), 416–433.

    Article  MathSciNet  MATH  Google Scholar 

  • Dragalin, V. and V. Fedorov (2006). Adaptive designs for dose-finding based on efficacy-toxicity response. J. Stat. Plann. Inference 136, 1800–1823.

    Article  MathSciNet  MATH  Google Scholar 

  • Elfving, G. (1952). Optimum allocation in linear regression. Ann. Math. Statist. 23, 255–262.

    Article  MathSciNet  MATH  Google Scholar 

  • Fedorov, V. (1971). The design of experiments in the multiresponse case. Theory Probab. Appl. 16(2), 323–332.

    Article  MATH  Google Scholar 

  • Fedorov, V. (1972). Theory of Optimal Experiments. New York: Academic Press.

    Google Scholar 

  • Fedorov, V. (1989). Optimal design with bounded density: optimization algorithms of the exchange type. J. Stat. Plann. Inference 22, 1–13.

    Article  MATH  Google Scholar 

  • Fedorov, V. and P. Hackl (1997). Model-Oriented Design of Experiments. Berlin: Springer.

    Book  MATH  Google Scholar 

  • Fedorov, V. and A. Pázman (1968). Design of physical experiments. Fortschritte der Physik 16, 325–355.

    Article  Google Scholar 

  • Fellman, J. (1974). On the allocation of linear observations. Comment. Phys. Math. 44, 27–78.

    MathSciNet  Google Scholar 

  • Galil, Z. and J. Kiefer (1977). Comparison of simplex designs for quadratic mixture models. Technometrics 19(4), 445–453.

    Article  MathSciNet  MATH  Google Scholar 

  • Hadjihassan, S., L. Pronzato, E. Walter, and I. Vuchkov (1997). Robust design for quality improvement by ellipsoidal bounding. In G. Yin and Q. Zhang (Eds.), Proc. 1996 AMS–SIAM Summer Seminar, Math. of Stochastic Manufacturing Systems, Lectures in Applied Math., vol. 33, Providence, Rhode Island, pp. 127–138. American Math. Soc.

    Google Scholar 

  • Harman, R. (2004a). Lower bounds on efficiency ratios based on ϕ p -optimal designs. In A. Di Bucchianico, H. Läuter, and H. Wynn (Eds.), mODa’7 – Advances in Model–Oriented Design and Analysis, Proc. 7th Int. Workshop, Heeze (Netherlands), Heidelberg, pp. 89–96. Physica Verlag.

    Google Scholar 

  • Harman, R. (2004b). Minimal efficiency of designs under the class of orthogonally invariant information criteria. Metrika 60, 137–153.

    Article  MathSciNet  MATH  Google Scholar 

  • Harman, R. and T. Jurík (2008). Computing c-optimal experimental designs using the simplex method of linear programming. Comput. Statist. Data Anal. 53, 247–254.

    Article  MathSciNet  MATH  Google Scholar 

  • Harman, R. and M. Trnovská (2009). Approximate D-optimal designs of experiments on the convex hull of a finite set of information matrices. Math. Slovaca 59(5), 693–704.

    Article  MathSciNet  MATH  Google Scholar 

  • Harville, D. (1997). Matrix Algebra from a Statistician’s Perspective. Heidelberg: Springer.

    Book  MATH  Google Scholar 

  • Karlin, S. and W. Studden (1966). Optimal experimental designs. Ann. Math. Statist. 37, 783–815.

    Article  MathSciNet  MATH  Google Scholar 

  • Khachiyan, L. (1979). A polynomial algorithm in linear programming. Doklady Akademïa Nauk SSSR 244, 1093–1096. [English transl. Soviet Math. Doklady, 20, 191–194].

    Google Scholar 

  • Khachiyan, L. (1996). Rounding of polytopes in the real number model of computation. Math. Oper. Res. 21(2), 307–320.

    Article  MathSciNet  MATH  Google Scholar 

  • Khachiyan, L. and M. Todd (1993). On the complexity of approximating the maximal inscribed ellipsoid for a polytope. Math. Programming A61(2), 137–159.

    Google Scholar 

  • Kiefer, J. (1962). Two more criteria equivalent to D-optimality of designs. Ann. Math. Statist. 33(2), 792–796.

    Article  MathSciNet  MATH  Google Scholar 

  • Kiefer, J. (1974). General equivalence theory for optimum designs (approximate theory). Ann. Statist. 2(5), 849–879.

    Article  MathSciNet  MATH  Google Scholar 

  • Kiefer, J. and J. Wolfowitz (1960). The equivalence of two extremum problems. Canad. J. Math. 12, 363–366.

    Article  MathSciNet  MATH  Google Scholar 

  • Liu, S. and H. Neudecker (1997). Kantorovich inequalities and efficiency comparisons for several classes of estimators in linear models. Statistica Neerlandica 51(3), 345–355.

    Article  MathSciNet  MATH  Google Scholar 

  • López-Fidalgo, J. and J. Rodríguez-Díaz (1998). Characteristic polynomial criteria in optimal experimental design. In A. Atkinson, L. Pronzato, and H. Wynn (Eds.), Advances in Model–Oriented Data Analysis and Experimental Design, Proc. MODA’5, Marseilles, June 22–26, 1998, pp. 31–38. Heidelberg: Physica Verlag.

    Google Scholar 

  • López-Fidalgo, J., B. Torsney, and R. Ardanuy (1998). MV-optimisation in weighted linear regression. In A. Atkinson, L. Pronzato, and H. Wynn (Eds.), Advances in Model–Oriented Data Analysis and Experimental Design, Proc. MODA’5, Marseilles, June 22–26, 1998, pp. 39–50. Heidelberg: Physica Verlag.

    Google Scholar 

  • Magnus, J. and H. Neudecker (1999). Matrix Differential Calculus, with Applications in Statistics and Econometrics. New York: Wiley.

    MATH  Google Scholar 

  • Mikulecká, J. (1983). On a hybrid experimental design. Kybernetika 19(1), 1–14.

    MathSciNet  MATH  Google Scholar 

  • Müller, C. and A. Pázman (1998). Applications of necessary and sufficient conditions for maximin efficient designs. Metrika 48, 1–19.

    MathSciNet  MATH  Google Scholar 

  • Pázman, A. (1986). Foundations of Optimum Experimental Design. Dordrecht (co-pub. VEDA, Bratislava): Reidel (Kluwer group).

    Google Scholar 

  • Pázman, A. (2001). Concentration sets, Elfving sets and norms in optimum design. In A. Atkinson, B. Bogacka, and A. Zhigljavsky (Eds.), Optimum Design 2000, Chapter 10, pp. 101–112. Dordrecht: Kluwer.

    Google Scholar 

  • Pázman, A. (2002a). Optimal design of nonlinear experiments with parameter constraints. Metrika 56, 113–130.

    Article  MathSciNet  Google Scholar 

  • Pečarić, J., S. Puntanen, and G. Styan (1996). Some further matrix extensions of the Cauchy-Schwarz and Kantorovich inequalities, with some statistical applications. Linear Algebra Appl. 237/238, 455–476.

    Google Scholar 

  • Pilz, J. (1983). Bayesian Estimation and Experimental Design in Linear Regression Models, Volume 55. Leipzig: Teubner-Texte zur Mathematik. [Also published by Wiley, New York, 1991].

    Google Scholar 

  • Polak, E. (1987). Optimization. Algorithms and Consistent Approximations. New York: Springer.

    Google Scholar 

  • Pronzato, L. (2004). A minimax equivalence theorem for optimum bounded design measures. Statist. Probab. Lett. 68, 325–331.

    Article  MathSciNet  MATH  Google Scholar 

  • Pronzato, L. (2006). On the sequential construction of optimum bounded designs. J. Stat. Plann. Inference 136, 2783–2804.

    Article  MathSciNet  MATH  Google Scholar 

  • Pronzato, L. (2009b). On the regularization of singular c-optimal designs. Math. Slovaca 59(5), 611–626.

    Article  MathSciNet  MATH  Google Scholar 

  • Pronzato, L. (2010b). Penalized optimal designs for dose-finding. J. Stat. Plann. Inference 140, 283–296.

    Article  MathSciNet  MATH  Google Scholar 

  • Pronzato, L., C.-Y. Huang, and E. Walter (1991). Nonsequential T-optimal design for model discrimination: new algorithms. In A. Pázman and J. Volaufová (Eds.), Proc. ProbaStat’91, Bratislava, pp. 130–136.

    Google Scholar 

  • Pronzato, L. and E. Walter (1994). Minimal-volume ellipsoids. Internat. J. Adapt. Control Signal Process. 8, 15–30.

    Article  MathSciNet  MATH  Google Scholar 

  • Pronzato, L. and E. Walter (1996). Volume-optimal inner and outer ellipsoids. In M. Milanese, J.-P. Norton, H. Piet-Lahanier, and E. Walter (Eds.), Bounding Approaches to System Identification, pp. 119–138. London: Plenum.

    Google Scholar 

  • Pronzato, L., H. Wynn, and A. Zhigljavsky (2000). Dynamical Search. Boca Raton: Chapman & Hall/CRC.

    MATH  Google Scholar 

  • Pronzato, L., H. Wynn, and A. Zhigljavsky (2005). Kantorovich-type inequalities for operators via D-optimal design theory. Linear Algebra Appl. 410, 160–169.

    Article  MathSciNet  MATH  Google Scholar 

  • Pukelsheim, F. (1993). Optimal Experimental Design. New York: Wiley.

    Google Scholar 

  • Raynaud, H.-F., L. Pronzato, and E. Walter (2000). Robust identification and control based on ellipsoidal parametric uncertainty descriptions. Eur. J. Control 6(3), 245–257.

    MathSciNet  Google Scholar 

  • Rényi, A. (1961). On measures of entropy and information. In Proc. 4th Berkeley Symp. on Math. Statist. and Prob., pp. 547–561.

    Google Scholar 

  • Rockafellar, R. (1970). Convex Analysis. Princeton: Princeton Univ. Press.

    MATH  Google Scholar 

  • Sahm, M. and R. Schwabe (2001). A note on optimal bounded designs. In A. Atkinson, B. Bogacka, and A. Zhigljavsky (Eds.), Optimum Design 2000, Chapter 13, pp. 131–140. Dordrecht: Kluwer.

    Google Scholar 

  • Schwabe, R. (1995). Designing experiments for additive nonlinear models. In C. Kitsos and W. Müller (Eds.), MODA4 – Advances in Model-Oriented Data Analysis, Spetses (Greece), june 1995, pp. 77–85. Heidelberg: Physica Verlag.

    Google Scholar 

  • Schwabe, R. (1996). Optimum Designs for Multi-Factor Models. New York: Springer.

    Book  MATH  Google Scholar 

  • Schwabe, R. (1997). Maximin efficient designs. Another view at D-optimality. Statist. Probab. Lett. 35, 109–114.

    Google Scholar 

  • Schwabe, R. and W. Wong (1999). Efficiency bounds for product designs in linear models. Ann. Inst. Statist. Math. 51, 723–730.

    Article  MathSciNet  MATH  Google Scholar 

  • Shor, N. and O. Berezovski (1992). New algorithms for constructing optimal circumscribed and inscribed ellipsoids. Optim. Methods Softw. 1, 283–299.

    Article  Google Scholar 

  • Sibson, R. (1972). Discussion on a paper by H.P. Wynn. J. Roy. Statist. Soc. B34, 181–183.

    Google Scholar 

  • Sibson, R. (1974). D A -optimality and duality. Progress in Stat. Colloq. Math. Soc. Janos Bolyai 9, 677–692.

    MathSciNet  Google Scholar 

  • Silvey, S. (1980). Optimal Design. London: Chapman & Hall.

    Book  MATH  Google Scholar 

  • Silvey, S. and D. Titterington (1973). A geometric approach to optimal design theory. Biometrika 60(1), 21–32.

    Article  MathSciNet  MATH  Google Scholar 

  • Studden, W. (1971). Elfving’s theorem and optimal designs for quadratic loss. Ann. Math. Statist. 42, 1613–1621.

    Article  MathSciNet  MATH  Google Scholar 

  • Sun, P. and R. Freund (2004). Computation of minimum-volume covering ellipsoids. Oper. Res. 52(5), 690–706.

    Article  MathSciNet  MATH  Google Scholar 

  • Tarasov, S., L. Khachiyan, and I. Erlich (1988). The method of inscribed ellipsoids. Soviet Math. Dokl. 37(1), 226–230.

    MathSciNet  MATH  Google Scholar 

  • Titterington, D. (1975). Optimal design: some geometrical aspects of D-optimality. Biometrika 62(2), 313–320.

    MathSciNet  MATH  Google Scholar 

  • Titterington, D. (1978). Estimation of correlation coefficients by ellipsoidal trimming. J. Roy. Statist. Soc. C27(3), 227–234.

    Google Scholar 

  • Todd, M. and E. Yildirim (2007). On Khachiyan’s algorithm for the computation of minimum volume enclosing ellipsoids. Discrete Appl. Math. 155, 1731–1744.

    Article  MathSciNet  MATH  Google Scholar 

  • Torsney, B. (1986). Moment inequalities via optimal design theory. Linear Algebra Appl. 82, 237–253.

    Article  MathSciNet  MATH  Google Scholar 

  • Vandenberghe, L., S. Boyd, and S.-P. Wu (1998). Determinant maximisation with linear matrix inequality constraints. SIAM J. Matrix Anal. Appl. 19(2), 499–533.

    Article  MathSciNet  MATH  Google Scholar 

  • Welzl, E. (1991). Smallest enclosing disks (balls and ellipsoids). In New Results and New Trends in Computer Science, Lecture Notes in Comput. Sci., vol. 555, pp. 359–370. Heidelberg: Springer.

    Google Scholar 

  • Wynn, H. (1977). Optimum designs for finite populations sampling. In S. Gupta and D. Moore (Eds.), Statistical Decision Theory and Related Topics II, pp. 471–478. New York: Academic Press.

    Google Scholar 

  • Wynn, H. (1982). Optimum submeasures with applications to finite population sampling. In S. Gupta and J. Berger (Eds.), Statistical Decision Theory and Related Topics III. Proc. 3rd Purdue Symp., vol. 2, pp. 485–495. New York: Academic Press.

    Google Scholar 

  • Yang, M. (2010). On de la Garza phenomenon. Ann. Statist. 38(4), 2499–2524.

    Article  MathSciNet  MATH  Google Scholar 

  • Yang, M. and J. Stufken (2009). Support points of locally optimal designs for nonlinear models with two parameters. Ann. Statist. 37(1), 518–541.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang, Y. and L. Gao (2003). On numerical solution of the maximum volume ellipsoid problem. SIAM J. Optim. 14(1), 53–76.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. French National Center for Scientific Research (CNRS), University of Nice, Sophia Antipolis, France

    Luc Pronzato

  2. Department of Applied Mathematics and Statistics, Comenius University, Bratislava, Slovakia

    Andrej Pázman

Authors
  1. Luc Pronzato
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrej Pázman
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this chapter

Cite this chapter

Pronzato, L., Pázman, A. (2013). Local Optimality Criteria Based on Asymptotic Normality. In: Design of Experiments in Nonlinear Models. Lecture Notes in Statistics, vol 212. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-6363-4_5

Download citation

Publish with us

Policies and ethics

Search

Navigation