Skip to main content
SpringerLink
Log in

Optimality of Unbiased Estimators: Nonasymptotic Theory

  • Chapter
  • First Online:
Mathematical Statistics

Part of the book series: Springer Series in Statistics ((PSS))

  • 3841 Accesses

Abstract

This essay describes the development of nonasymptotic optimality concepts for estimators, in particular of bounds on the risk of mean unbiased estimators for the quadratic loss function and, more generally, for convex loss functions. We present the result of Rao, Blackwell, Lehmann and Scheffé that the conditional expectation of a mean unbiased estimator with respect to a sufficient and complete statistic minimizes the convex risk among all mean unbiased estimators. We also describe Bahadur’s converse: If for every unbiasedly estimable functional there is a quadratically optimal unbiased estimator, then there exists a sufficient sub-\(\sigma \)-field. We discuss the history of the Cramér–Rao bound. Finally, bounds for the concentration of median unbiased estimators and of confidence procedures are given.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Aitken, A. C., & Silverstone, H. (1942). On the estimation of statistical parameters. Proceedings of the Royal Society of Edinburgh Section A, 61, 186–194.

    Article  MATH  MathSciNet  Google Scholar 

  • Andersen, E. B. (1970). Asymptotic properties of conditional maximum-likelihood estimators. Journal of the Royal Statistical Society. Series B, 32, 283–301.

    MATH  Google Scholar 

  • Bahadur, R. R. (1957). On unbiased estimates of uniformly minimum variance. Sankhyā, 18, 211–224.

    MATH  MathSciNet  Google Scholar 

  • Barankin, E. W. (1949). Locally best unbiased estimates. The Annals of Mathematical Statistics, 20, 477–501.

    Article  MATH  MathSciNet  Google Scholar 

  • Barankin, E. W. (1950). Extension of a theorem of Blackwell. The Annals of Mathematical Statistics, 21, 280–284.

    Article  MATH  MathSciNet  Google Scholar 

  • Barankin, E. W. (1951). Conditional expectation and convex functions. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability (pp. 167–169). Berkeley: University of California Press.

    Google Scholar 

  • Barnett, V. (1975). Comparative statistical inference. New York: Wiley.

    MATH  Google Scholar 

  • Barton, D. E. (1961). Unbiased estimation of a set of probabilities. Biometrika, 48, 227–229.

    Article  MATH  MathSciNet  Google Scholar 

  • Basu, A. P. (1964). Estimators of reliability for some distributions useful in life testing. Technometrics, 6, 215–219.

    Article  Google Scholar 

  • Basu, D. (1955). A note on the theory of unbiased estimation. The Annals of Mathematical Statistics, 26, 345–348.

    Article  MATH  MathSciNet  Google Scholar 

  • Bednarek-Kozek, B., & Kozek, A. (1978). Two examples of strictly convex non-universal loss functions (p. 133). Preprint: Institute of Mathematics Polish Academy of Sciences.

    MATH  Google Scholar 

  • Bickel, P. J., Klaassen, C. A. J., Ritov, Y., & Wellner, J. A. (1993). Efficient and adaptive estimation for semiparametric models. Baltimore: Johns Hopkins University Press (1998 Springer Paperback).

    Google Scholar 

  • Birnbaum, A. (1961). A unified theory of estimation, I. The Annals of Mathematical Statistics, 32, 112–135.

    Article  MATH  Google Scholar 

  • Birnbaum, A. (1964). Median-unbiased estimators. Bulletin of Mathematics and Statistics, 11, 25–34.

    MATH  MathSciNet  Google Scholar 

  • Blackwell, D. (1947). Conditional expectation and unbiased sequential estimation. The Annals of Mathematical Statistics, 18, 105–110.

    Article  MATH  MathSciNet  Google Scholar 

  • Bomze, I. M. (1986). Measurable supports, reducible spaces and the structure of the optimal \(\sigma \)-field in unbiased estimation. Monatshefte für Mathematik, 101, 27–38.

    Article  MATH  MathSciNet  Google Scholar 

  • Bomze, I. M. (1990). A functional analytic approach to statistical experiments (Vol. 237). Pitman research notes in mathematics series. Harlow: Longman.

    Google Scholar 

  • Borges, R., & Pfanzagl, J. (1963). A characterization of the one-parameter exponential family of distributions by monotonicity of likelihood ratios. Z. Wahrscheinlichkeitstheorie verw. Gebiete, 2, 111–117.

    Article  MATH  MathSciNet  Google Scholar 

  • Brown, L. D., Cohen, A., & Strawderman, W. E. (1976). A complete class theorem for strict monotone likelihood ratio with applications. The Annals of Statistics, 4, 712–722.

    Article  MATH  MathSciNet  Google Scholar 

  • Chapman, D. G., & Robbins, H. (1951). Minimum variance estimation without regularity assumptions. The Annals of Mathematical Statistics, 22, 581–586.

    Article  MATH  MathSciNet  Google Scholar 

  • Clopper, C. J., & Pearson, E. S. (1934). The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika, 26, 404–413.

    Article  MATH  Google Scholar 

  • Cournot, A. A. (1843). Exposition de la théorie des chances et des probabilités. Paris: Hachette.

    MATH  Google Scholar 

  • Cramér, H. (1946). Mathematical methods of statistics. Princeton: Princeton University Press.

    MATH  Google Scholar 

  • Darmois, G. (1945). Sur les lois limites de la dispersion de certain estimations. Revue de l’Institut International de Statistique, 13, 9–15.

    Article  Google Scholar 

  • Eberl, W. (1984). On unbiased estimation with convex loss functions. In E. J. Dudewitz, D. Plachky, & P. K. Sen (Eds.), Recent results in estimation theory and related topics (pp. 177–192). Statistics and Decisions, Supplement, Issue 1.

    Google Scholar 

  • Eberl, W., & Moeschlin, O. (1982). Mathematische Statistik. Berlin: De Gruyter.

    MATH  Google Scholar 

  • Fisher, R. A. (1930). Inverse probability. Mathematical Proceedings of the Cambridge Philosophical Society, 26, 528–535.

    Google Scholar 

  • Fisz, M. (1963). Probability theory and mathematical statistics (3rd ed.). (1st ed. 1954 in Polish, 2nd ed. 1958 in Polish and German). New York: Wiley.

    Google Scholar 

  • Fourier, J. B. J. (1826). Mémoire sur les résultats moens déduits d’un grand nombre d’observations. Recherches statistiques sur la ville de Paris et le département de la Seine. Reprinted in Œuvres, 2, 525–545.

    Google Scholar 

  • Fréchet, M. (1943). Sur l’extension de certaines évaluations statistiques de petits échantillons. Revue de l’Institut International de Statistique, 11, 182–205.

    Article  MATH  Google Scholar 

  • Gauss, C. F. (1816). Bestimmung der Genauigkeit der Beobachtungen. Carl Friedrich Gauss Werke 4, 109–117, Königliche Gesellschaft der Wissenschaften, Göttingen.

    Google Scholar 

  • Ghosh, J. K. (1961). On the relation among shortest confidence intervals of different types. Calcutta Statistical Association Bulletin, 10, 147–152.

    Article  MATH  MathSciNet  Google Scholar 

  • Halmos, P. R. (1946). The theory of unbiased estimation. The Annals of Mathematical Statistics, 17, 34–43.

    Article  MATH  MathSciNet  Google Scholar 

  • Hammersley, J. M. (1950). On estimating restricted parameters. Journal of the Royal Statistical Society. Series B, 12, 192–229; Discussion, 230–240.

    Google Scholar 

  • Heizmann, H.-H. (1989). UMVU-Schätzer und ihre Struktur. Systems in Economics, 112. Athenäum-Verlag.

    Google Scholar 

  • Heyer, H. (1973). Mathematische Theorie statistischer Experimente. Hochschultext. Berlin: Springer.

    Book  MATH  Google Scholar 

  • Heyer, H. (1982). Theory of statistical experiments. Springer series in statistics. Berlin: Springer.

    Book  MATH  Google Scholar 

  • Hodges, J. L, Jr., & Lehmann, E. L. (1950). Some problems in minimax point estimation. The Annals of Mathematical Statistics, 21, 182–197.

    Article  MATH  MathSciNet  Google Scholar 

  • Hotelling, H. (1931). The generalization of Student’s ratio. The Annals of Mathematical Statistics, 2, 360–378.

    Article  MATH  Google Scholar 

  • Kolmogorov, A. N. (1950). Unbiased estimators. Izvestiya Akademii Nauk S.S.S.R. Seriya Matematicheskaya, 14, 303–326. Translation: pp. 369–394 in: Selected works of A.N. Kolmogorov (Vol. II). Probability and mathematical statistics. A. N. Shiryaev (ed.), Mathematics and its applications (Soviet series) (Vol. 26). Dordrecht: Kluwer Academic (1992).

    Google Scholar 

  • Kozek, A. (1977). On the theory of estimation with convex loss functions. In R. Bartoszynski, E. Fidelis, & W. Klonecki (Eds.), Proceedings of the Symposium to Honour Jerzy Neyman (pp. 177–202). Warszawa: PWN-Polish Scientific Publishers.

    Google Scholar 

  • Kozek, A. (1980). On two necessary and sufficient \(\sigma \)-fields and on universal loss functions. Probability and Mathematical Statistics, 1, 29–47.

    MATH  MathSciNet  Google Scholar 

  • Krámli, A. (1967). A remark to a paper of L. Schmetterer. Studia Scientiarum Mathematicarum Hungarica, 2, 159–161.

    MATH  Google Scholar 

  • Laplace, P. S. (1812). Théorie analytique des probabilités. Paris: Courcier.

    Google Scholar 

  • Lehmann, E. L. (1959). Testing statistical hypotheses. New York: Wiley.

    MATH  Google Scholar 

  • Lehmann, E. L., & Casella, G. (1998). Theory of point estimation (2nd ed.). Berlin: Springer.

    Google Scholar 

  • Lehmann, E. L., & Scheffé, H. (1950, 1955, 1956). Completeness, similar regions and unbiased estimation. Sankhyā, 10, 305–340; 15, 219–236; Correction, 17 250.

    Google Scholar 

  • Lexis, W. (1875). Einleitung in die Theorie der Bevölkerungsstatistik. Straßburg: Trübner.

    Google Scholar 

  • Linnik, Yu V, & Rukhin, A. L. (1971). Convex loss functions in the theory of unbiased estimation. Soviet Mathematics Doklady, 12, 839–842.

    MATH  Google Scholar 

  • Lumel’skii, Ya. P., & Sapozhnikov, P. N., (1969). Unbiased estimates of density functions. Theory of Probability and Its Applications, 14, 357–364.

    Google Scholar 

  • Müller-Funk, U., Pukelsheim, F., & Witting, H. (1989). On the attainment of the Cramér-Rao bound in \(L_r\)-differentiable families of distributions. The Annals of Statistics, 17, 1742–1748.

    Article  MATH  MathSciNet  Google Scholar 

  • Neyman, J. (1934). On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection. The Journal of the Royal Statistical Society, 97, 558–625.

    Google Scholar 

  • Neyman, J. (1937). Outline of a theory of statistical estimation based on the classical theory of probability. Philosophical Transactions of the Royal Society of London. Series A, 236, 333–380.

    Google Scholar 

  • Neyman, J. (1938). Léstimation statistique traitée comme un probléme classique de probabilité. Actualités Sci. Indust., 739, 25–57.

    Google Scholar 

  • Neyman, J. (1941). Fiducial argument and the theory of confidence intervals. Biometrika, 32, 128–150.

    Article  MATH  MathSciNet  Google Scholar 

  • Padmanabhan, A. R. (1970). Some results on minimum variance unbiased estimation. Sankhyā Ser. A, 32, 107–114.

    MATH  Google Scholar 

  • Pfanzagl, J. (1970). Median-unbiased estimators for M.L.R.-families. Metrika, 15, 30–39.

    Article  MATH  MathSciNet  Google Scholar 

  • Pfanzagl, J. (1972). On median-unbiased estimates. Metrika, 18, 154–173.

    Article  MATH  MathSciNet  Google Scholar 

  • Pfanzagl, J. (1979). On optimal median-unbiased estimators in the presence of nuisance parameters. The Annals of Statistics, 7, 187–193.

    Article  MATH  MathSciNet  Google Scholar 

  • Pfanzagl, J. (1993). Sequences of optimal unbiased estimators need not be asymptotically optimal. Scandinavian Journal of Statistics, 20, 73–76.

    MATH  MathSciNet  Google Scholar 

  • Pfanzagl, J. (1994). Parametric statistical theory. Berlin: De Gruyter.

    Book  MATH  Google Scholar 

  • Polfeldt, T. (1970). Asymptotic results in non-regular estimation. Skand. Aktuarietidskr. Supplement 1–2.

    Google Scholar 

  • Portnoy, S. (1977). Asymptotic efficiency of minimum variance unbiased estimators. The Annals of Statistics, 5, 522–529.

    Article  MATH  MathSciNet  Google Scholar 

  • Pratt, J. W. (1961). Length of confidence intervals. Journal of the American Statistical Association, 56, 549–567.

    Article  MATH  MathSciNet  Google Scholar 

  • Rao, C. R. (1945). Information and accuracy attainable in the estimation of statistical parameters. Bulletin of Calcutta Mathematical Society, 37, 81–91.

    MATH  MathSciNet  Google Scholar 

  • Rao, C. R. (1947). Minimum variance estimation of several parameters. Proceedings of the Cambridge Philosophical Society, 43, 280–283.

    Article  MATH  MathSciNet  Google Scholar 

  • Rao, C. R. (1949). Sufficient statistics and minimum variance estimates. Proceedings of the Cambridge Philosophical Society, 45, 213–218.

    Article  MATH  MathSciNet  Google Scholar 

  • Rao, C. R. (1952). Some theorems on minimum variance estimation. Sankhyā, 12, 27–42.

    MATH  MathSciNet  Google Scholar 

  • Savage, L. J. (1954). The foundations of statistics. New York: Wiley.

    MATH  Google Scholar 

  • Schmetterer, L. (1960). On unbiased estimation. The Annals of Mathematical Statistics, 31, 1154–1163.

    Article  MATH  MathSciNet  Google Scholar 

  • Schmetterer, L. (1966). On the asymptotic efficiency of estimates. In Research Papers in Statistics. Festschrift for J. Neyman & F. N. David (Eds.) (pp. 301–317). New York: Wiley.

    Google Scholar 

  • Schmetterer, L. (1974). Introduction to mathematical statistics. Translation of the 2nd German edition 1966. Berlin: Springer.

    Google Scholar 

  • Schmetterer, L. (1977). Einige Resultate aus der Theorie erwartungstreuer Schätzungen. In Transactions of the Seventh Prague Conference on Information Theory, Statistical Decision Functions, Random processes (Vol. B, pp. 489–503).

    Google Scholar 

  • Schmetterer, L., & Strasser, H. (1974). Zur Theorie der erwartungstreuen Schätzungen. Anzeiger, Österreichische Akadademie der Wissenschaften, Mathematisch-Naturwissenschaftliche. Klasse, 6, 59–66.

    Google Scholar 

  • Seheult, A. H., & Quesenberry, C. P. (1971). On unbiased estimation of density functions. The Annals of Mathematical Statistics, 42, 1434–1438.

    Article  MATH  Google Scholar 

  • Shao, J. (2003). Mathematical statistics (2nd ed.). Berlin: Springer.

    Book  MATH  Google Scholar 

  • Simons, G., & Woodroofe, M. (1983). The Cramér-Rao inequality holds almost everywhere. In M. H. Rizvi, J. S. Rustagi, & D. Siegmund (Eds.), Papers in Honor of Herman Chernoff (pp. 69–83). New York: Academic Press.

    Google Scholar 

  • Stein, Ch. (1950). Unbiased estimates of minimum variance. The Annals of Mathematical Statistics, 21, 406–415.

    Article  MATH  MathSciNet  Google Scholar 

  • Strasser, H. (1972). Sufficiency and unbiased estimation. Metrika, 19, 98–114.

    Article  MATH  MathSciNet  Google Scholar 

  • Strasser, H. (1985). Mathematical theory of statistics. Berlin: De Gruyter.

    Book  MATH  Google Scholar 

  • Stuart, A., & Ord, K. (1991). Kendall’s advanced theory of statistics. Classical inference and relationship (5th ed., Vol. 2). London: Edward Arnold.

    Google Scholar 

  • Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209–212.

    Article  Google Scholar 

  • Witting, H. (1985). Mathematische Statistik I. Parametrische Verfahren bei festem Stichprobenumfang: Teubner.

    Book  MATH  Google Scholar 

  • Witting, H., & Müller-Funk, U. (1995). Mathematische Statistik II. Asymptotische Statistik: Parametrische Modelle und nichtparametrische Funktionale. Teubner.

    Book  MATH  Google Scholar 

  • Working, H., & Hotelling, H. (1929). Applications of the theory of error to the interpretation of trends. Journal of the American Statistical Association, 24(165A), 73–85.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mathematical Institute, University of Cologne, Cologne, Germany

    Johann Pfanzagl

Authors
  1. Johann Pfanzagl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Johann Pfanzagl .

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Pfanzagl, J. (2017). Optimality of Unbiased Estimators: Nonasymptotic Theory. In: Mathematical Statistics. Springer Series in Statistics(). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31084-3_4

Download citation

Publish with us

Policies and ethics

Search

Navigation