The Development of a Frequentist Error Theory

Abstract

Gauss did not take the trouble to rewrite his first proof of the method of least squares in terms of direct probability. This task was carried out by astronomers and geodesists writing elementary textbooks on the method of least squares. They found Gauss’s second proof too cumbersome for their readers and did not need the generalization involved because the measurement errors encountered in their fields were in most cases nearly normally distributed. As far as error theory is concerned, they realized that the principle of inverse probability was superfluous. The method of maximizing the posterior density could be replaced by the method of maximizing the density p(x|θ) of the observations, which would lead to the same estimates because p(x|θ) α p(θ|x). This method has an obvious intuitive appeal and goes back to Daniel Bernoulli and Lambert; see Todhunter [261], pp. 236–237) and Edwards [50]. Todhunter writes:

Thus Daniel Bernoulli agrees in some respects with modern theory. The chief difference is that modern theory takes for the curve of probability that defined by the equation

$$ y = \sqrt {c/\pi e^{ - cx^2 } ,} $$

while Daniel Bernoulli takes a [semi]circle.