Condition Numbers and Least Squares Regression

Abstract

Many problems in geometric modelling require the approximation of a set of data points by a weighted linear combination of basis functions. This yields an over-determined linear algebraic equation, which is usually solved in the least squares (LS) sense. The numerical solution of this problem requires an estimate of its condition number, of which there are several. These condition numbers are considered theoretically and computationally in this paper, and it is shown that they include a simple normwise measure that may overestimate by several orders of magnitude the true numerical condition of the LS problem, to refined componentwise and normwise measures. Inequalities that relate these condition numbers are established, and it is concluded that the solution of the LS problem may be well-conditioned in the normwise sense, even if one of its components is ill-conditioned. An example of regression using radial basis functions is used to illustrate the differences in the condition numbers.