Model Assessment and Selection in Multiple Regression
Regression, as a scientific method, first appeared around 1885, although the method of least squares was discovered 80 years earlier. Least squares owes its origins to astronomy and, specifically, to Legendre’s 1805 pioneering work on the determination of the orbits of planets in which he introduced and named the method of least squares. Adrien Marie Legendre estimated the coefficients of a set of linear equations by minimizing the error sum of squares. Gauss stated in 1809 that he had been using the method since 1795, but could not prove his claim with documented evidence. Within a few years, Gauss and Pierre Simon Laplace added a probability component — a Gaussian curve to describe the error distribution — that was crucial to the success of the method. Gauss went on to devise an elimination algorithm to compute least-squares estimates. Once introduced, least squares caught on immediately in astronomy and geodetics, but it took 80 years for these ideas to be transported to other disciplines.
KeywordsPenalty Function Ridge Regression Principal Component Regression Shrinkage Estimator Ridge Parameter
Unable to display preview. Download preview PDF.