Advertisement

Statistical Papers

, Volume 60, Issue 2, pp 195–214 | Cite as

Bregman divergences based on optimal design criteria and simplicial measures of dispersion

  • Luc PronzatoEmail author
  • Henry P. Wynn
  • Anatoly Zhigljavsky
Regular Article
  • 33 Downloads

Abstract

In previous work the authors defined the k-th order simplicial distance between probability distributions which arises naturally from a measure of dispersion based on the squared volume of random simplices of dimension k. This theory is embedded in the wider theory of divergences and distances between distributions which includes Kullback–Leibler, Jensen–Shannon, Jeffreys–Bregman divergence and Bhattacharyya distance. A general construction is given based on defining a directional derivative of a function \(\phi \) from one distribution to the other whose concavity or strict concavity influences the properties of the resulting divergence. For the normal distribution these divergences can be expressed as matrix formula for the (multivariate) means and covariances. Optimal experimental design criteria contribute a range of functionals applied to non-negative, or positive definite, information matrices. Not all can distinguish normal distributions but sufficient conditions are given. The k-th order simplicial distance is revisited from this aspect and the results are used to test empirically the identity of means and covariances.

Keywords

Simplicial distances Bregman divergence Optimal design criteria Burbea-Rao divergence Energy statistic 

Mathematics Subject Classification

62H30 62K05 

Notes

References

  1. Atkinson AC, Donev AN, Tobias RD (2007) Optimum experimental designs, with SAS. Oxford University Press, OxfordzbMATHGoogle Scholar
  2. Basseville M (2013) Divergence measures for statistical data processing—an annotated bibliography. Signal Process 93(4):621–633MathSciNetCrossRefGoogle Scholar
  3. Bhattacharyya A (1946) On a measure of divergence between two multinomial populations. Sankhyā 7(4):401–406MathSciNetzbMATHGoogle Scholar
  4. Björck G (1956) Distributions of positive mass, which maximize a certain generalized energy integral. Arkiv för Matematik 3(21):255–269MathSciNetCrossRefzbMATHGoogle Scholar
  5. Fedorov VV (1972) Theory of optimal experiments. Academic Press, New YorkGoogle Scholar
  6. Fedorov VV, Hackl P (1997) Model-oriented design of experiments. Springer, BerlinCrossRefzbMATHGoogle Scholar
  7. Fedorov VV, Leonov SL (2014) Optimal design for nonlinear response models. CRC Press, Boca RatonzbMATHGoogle Scholar
  8. Frigyik BA, Srivastava S, Gupta MR (2008) Functional Bregman divergence and Bayesian estimation of distributions. IEEE Trans Inf Theory 54(11):5130–5139MathSciNetCrossRefzbMATHGoogle Scholar
  9. Kiefer J (1974) General equivalence theory for optimum designs (approximate theory). Ann Stat 2(5):849–879MathSciNetCrossRefzbMATHGoogle Scholar
  10. López-Fidalgo J, Rodríguez-Díaz JM (1998) Characteristic polynomial criteria in optimal experimental design. In: Atkinson AC, Pronzato L, Wynn HP (eds) Advances in model-oriented data analysis and experimental design. Proceedings of MODA’5, Marseilles. Physica Verlag, Heidelberg, June 22–26, pp 31–38Google Scholar
  11. Łukaszyk S (2004) A new concept of probability metric and its applications in approximation of scattered data sets. Comput Mech 33(4):299–304MathSciNetCrossRefzbMATHGoogle Scholar
  12. Nielsen F, Boltz S (2011) The Burbea-Rao and Bhattacharyya centroids. IEEE Trans Inf Theory 57(8):5455–5466MathSciNetCrossRefzbMATHGoogle Scholar
  13. Nielsen F, Nock R (2017) Generalizing Jensen and Bregman divergences with comparative convexity and the statistical Bhattacharyya distances with comparable means. arXiv preprint arXiv:1702.04877
  14. Pázman A (1986) Foundations of optimum experimental design. Kluwer group, co-pub. VEDA, Bratislava, Reidel, DordrechtzbMATHGoogle Scholar
  15. Pronzato L, Pázman A (2013) Design of experiments in nonlinear models. Asymptotic normality, optimality criteria and small-sample properties. Springer, LNS 212, New YorkzbMATHGoogle Scholar
  16. Pronzato L, Wynn HP, Zhigljavsky A (2016) Extremal measures maximizing functionals based on simplicial volumes. Stat Pap 57(4):1059–1075 hal-01308116MathSciNetCrossRefzbMATHGoogle Scholar
  17. Pronzato L, Wynn HP, Zhigljavsky A (2017) Extended generalised variances, with applications. Bernoulli 23(4A):2617–2642MathSciNetCrossRefzbMATHGoogle Scholar
  18. Pronzato L, Wynn HP, Zhigljavsky AA (2018) Simplicial variances, potentials and Mahalanobis distances. J Multivar Anal (to appear)Google Scholar
  19. Pukelsheim F (1993) Optimal experimental design. Wiley, New YorkzbMATHGoogle Scholar
  20. Rodríguez-Díaz JM, López-Fidalgo J (2003) A bidimensional class of optimality criteria involving \(\phi _p\) and characteristic criteria. Statistics 37(4):325–334MathSciNetCrossRefzbMATHGoogle Scholar
  21. Schilling RL, Song R, Vondracek Z (2012) Bernstein functions: theory and applications. de Gruyter, BerlinCrossRefzbMATHGoogle Scholar
  22. Sejdinovic S, Sriperumbudur B, Gretton A, Fukumizu K (2013) Equivalence of distance-based and RKHS-based statistics in hypothesis testing. Ann Stat 41(5):2263–2291MathSciNetCrossRefzbMATHGoogle Scholar
  23. Shiryaev AN (1996) Probability. Springer, BerlinCrossRefzbMATHGoogle Scholar
  24. Silvey SD (1980) Optimal design. Chapman & Hall, LondonCrossRefzbMATHGoogle Scholar
  25. Sriperumbudur BK, Gretton A, Fukumizu K, Schölkopf B, Lanckriet GRG (2010) Hilbert space embeddings and metrics on probability measures. J Mach Learn Res 11(Apr):1517–1561MathSciNetzbMATHGoogle Scholar
  26. Székely GJ, Rizzo ML (2013) Energy statistics: a class of statistics based on distances. J Stat Plan Inference 143(8):1249–1272MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.CNRS, UCA, Laboratoire I3S, UMR 7172; 2000, route des Lucioles, Les AlgorithmesSophia AntipolisFrance
  2. 2.London School of EconomicsLondonUK
  3. 3.School of MathematicsCardiff UniversityCardiffUK

Personalised recommendations