Improving on Standard Bayesian and Frequentist Estimators

Abstract

Suppose that an experiment of interest will yield the outcome X, where X has been modeled as having the probability distribution Fq depending on the scalar parameter q. The statistician is prepared to estimate q by the estimator bq = bq(X), where bq might be either a Bayesian or a frequentist estimator, depending on the statistician’s inclination. Suppose that before this estimation process is completed, the statistician becomes aware of the outcome Y of a “similar“ experiment. The question then naturally arises: can the information obtained in the other experiment be exploited to provide a better estimator of q than bq? If it can, then the opportunity afforded to the statistician to improve upon his initial strategy should not be squandered! The situation I have outlined here is both intriguing and seductive, but it is also vague, involving the as-yet-undefined term “similar“ as well as dependent on suppositions that might, in many circumstances, be found to be unrealistic. In this chapter, we will examine a scenario in which opportunities of the sort above tend to arise. The empirical Bayes framework, the quintessential statistical setting in which one may reliably learn from similar experiments, was introduced by Robbins at the Berkeley Symposium on Probability and Statistics in 1955, and was developed further in Robbins (1964). This section is aimed at presenting the empirical Bayes “model“ and discussing its statistical implications.