Robust Regression

Abstract

Consider the multivariate linear model

$$ Y_i = X'_i \beta + E_i ,{\text{ }}i = 1,...,n, $$

(9.1)

where Y_i : p × 1 is the observation on the i^th individual, X_i : q × p is the design matrix with known elements, β : q × 1 is a vector of unknown regression coefficients, and E_i : p × 1 is the unobservable random error that is usually assumed to be suitably centered and to have a p-variate distribution. A central problem in linear models is estimating the regression vector β. Note that model (9.1) reduces to the univariate regression model when p = 1, which we can write as

$$ y_i = x'_i \beta + \varepsilon _i ,{\text{ }}i = 1,...,n, $$

(9.2)

where x_i is now a q-vector. Model (9.1) becomes the classical multivariate regression, also called MANOVA model, when X_i : q × p is of the special form

$$ X_i = \left( {\begin{array}{*{20}c} {x_i } \\ 0 \\ \vdots \\ 0 \\ \end{array} \begin{array}{*{20}c} 0 \\ {x_i } \\ \vdots \\ 0 \\ \end{array} \begin{array}{*{20}c} \cdots \\ \cdots \\ \cdots \\ \cdots \\ \end{array} \begin{array}{*{20}c} 0 \\ 0 \\ \vdots \\ {x_i } \\ \end{array} } \right), $$

(9.3)

where x_i : m × 1 and q = mp. Our discussion of the general model will cover both classical cases considered in the literature.