Multivariate regression

Abstract

Multivariate regression with p responses as opposed to p multiple regressions is getting increasingly more attention, especially in the context of prediction. In this chapter, we generalize the multiple regression model of Section 5.6.2 to the multivariate case. The estimation method of Section 9.2 relies also on orthogonal projections. The model considered is

$$ {\text{Y}} = {\text{XB}} + {\text{E}}, $$

where Y ∈ ℝ ^sun_inp , B ∈ ℝ ^suk_inp and X ∈ ℝ ^sun_ink of rank X = k is fixed. The error term E is such that E E = 0 and var E = I _n ⊗ Σ with Σ > 0 in ℝ ^sup_inp . The observation vectors consisting of the rows of Y are thus uncorrelated. The Gauss-Markov estimate is derived first. Then, assuming normality, the maximum likelihood estimates of B and Σ are obtained together with the fundamental result about their joint distribution. Section 9.3 derives the likelihood ratio test for the general linear hypothesis

$$ H_0 :CB = 0 $$

against all alternatives where C ∈ ℝ ^sur_ink of rank C = r in the above model. In the last sections, we discuss the practical and more commonly encountered situation of k random (observed) predictors and the problem of prediction of p responses from the same set of k predictors. Finally, an application to the MANOVA one-way classification model is treated as a special case.