Abstract
The method of least squares is the standard method for finding an approximate solution of a linear system with more equations than unknowns. Such systems occur in data fitting where one determines the fit by minimizing the sum of squares of the difference between the observed and the fitted values. The linear least squares problem also occurs in statistical regression analysis, in signal processing, and in many other applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Show first that \(\langle \boldsymbol{b}_{\rm{max}},\boldsymbol{v}_{j}\rangle =\langle \boldsymbol{b},\boldsymbol{v}_{j}\rangle\) for j = 1, 2, …, k.
- 2.
Show that if \({\boldsymbol{b}_{\rm{max}}}_{1}\) and \({\boldsymbol{b}_{\rm{max}}}_{2}\) both satisfies (10.2) then \({\boldsymbol{b}_{\rm{max}}}_{1} = {\boldsymbol{b}_{\rm{max}}}_{2}\).
- 3.
Suppose \(\boldsymbol{h} = \boldsymbol{f}_{1} + \boldsymbol{g}_{1} = \boldsymbol{f}_{2} + \boldsymbol{g}_{2}\). Show that \(\boldsymbol{f}_{1} -\boldsymbol{f}_{2} \in {\rm{max}}\cap \mathcal{G}\).
- 4.
If \(\boldsymbol{f} \in \text{span}(\boldsymbol{A})\) then \(\boldsymbol{f} = \boldsymbol{A}\boldsymbol{x}\) for some \(\boldsymbol{x}\).
- 5.
Let \(\boldsymbol{b} \in \mathbb{R}^{m}\). Define \(\boldsymbol{b}_{1}\) as the orthogonal projection of \(\boldsymbol{b}\) onto \(\text{span}(\boldsymbol{A})\) and use (10.2).
- 6.
Use (10.1) to find \(\boldsymbol{b}_{1}\).
- 7.
Use the fact that \({\rm{max}}\cap \mathcal{G} =\{ \boldsymbol{0}\}\).
- 8.
The least squares solution is \(\boldsymbol{x} = \boldsymbol{A}^{\dag }\boldsymbol{b}\).
- 9.
Use (10.3).
- 10.
Note the ordering of the coefficient of a polynomial in MATLAB. For example \(p(x) =\alpha _{2} +\alpha _{1}x\) in the linear case.
- 11.
To compute x 2 use the instruction x.*x.
- 12.
Vandermonde matrix.
- 13.
If \(\boldsymbol{u} = (u_{i-n_{\ell}},\ldots,u_{i},\ldots,u_{i+n_{r}})^{T} \in \mathbb{R}^{m}\) then \(\|\boldsymbol{u}\|_{2} = \left (\sum _{j=i-n_{\ell}}^{i+n_{r}}u_{ j}^{2}\right )^{1/2}\).
References
J. Bastien, Introduction à l’analyse numérique: Applications sous MATLAB, Dunod, 2003.
A. Biran, M. Breiner, MATLAB6 for Engineers, Prentice Hall, 2002, 3th ed.
A. Björck, Numerical Methods for Least Squares Problems, SIAM, 1996.
K. Chen, P. Giblin, A. Irving, Mathematical Explorations with MATLAB, Cambrige Univ. Press, 1999.
E. Cohen, R. F. Riesenfeld, G. Elber, Geometric Modeling with Splines: An Introduction, A.K. Peters, Ltd., 2001.
J. Cooper, MATLAB Companion for Multivariable Calculus (a), Academic Press, 2001.
M. Crouzeix, A. Mignot, Analyse numérique des équations différentielles, Masson, 1989.
C. de Boor, A Practical Guide to Splines, Springer-Verlag, 2001, Rev. Ed.
P. Deuflhard, A. Hohmann, Numerical Analysis in Modern Scientific Computing, An Introduction, Springer-Verlag, 2003, 2nd ed.
A. Fortin, Analyse numérique pour ingénieurs, Ed. de l’Ecole Polytechnique de Montréal, 1995.
W. Gander, J. Hřebíček, Solving Problems in Scientific Computing using MAPPLE and MATLAB, Springer-Verlag, 2004, 4th ed.
S. Godounov, V. Riabenki, Schémas aux différences, MIR, 1977.
G.H. Golub, C.F. Van Loan, Matrix Computations, The Johns Hopkins Univ. Press, 1996, 3rd Ed.
A. Greenbaum, Iterative mMethods for Solving Linear Systems, SIAM, 1997.
F. Gustafsson, N. Bergman, MATLAB for Engineers Explained, Springer-Verlag, 2003.
D.J. Higham, N. Higham, MATLAB Guide, SIAM, 2005, 2nd Ed.
A. Kharab, R. B. Guenther, Introduction to Numerical Methods (a): a MATLAB Approach, Chapman and Hall, 2002.
P. Lascaux and R. Théodor, Analyse numérique matricielle appliquée à l’art de l’ingénieur, vol 1 and 2, Dunod, 2004.
A.J. Laub, Matrix Analysis for Scientists and Engineers, SIAM, 2005.
P. Linz, R. Wang, Exploring Numerical Methods: An Introduction to Scientific Computing using MATLAB, Jones and Barlett Pub., 2003.
D. Marsh, Applied Geometry for Computer Graphics and CAD, Springer-Verlag, 1999.
C. B. Moler, Numerical Computing with MATLAB, SIAM, 2004.
G.M. Phillips, Interpolation and Approximation by Polynomials, CMS Books in Mathematics, Springer-Verlag 2003.
A. Quarteroni, R. Sacco, F. Saleri Numerical Mathematics, Springer-Verlag, 2000.
A. Quarteroni, R. Sacco, F. Saleri Scientific Computing with MATLAB, Springer-Verlag, 2003.
D. Salomon, Curves and Surfaces for Computer Graphics, Springer-Verlag, 2006.
M. Schatzman, Numerical Analysis: A Mathematical Introduction, Oxford Univ. Press, 2002.
B.J. Schroers, Ordinary Differential Equations: a Practical Guide, Cambridge Univ. Press, 2011.
E. Süli, D. Mayers, An introduction to Numerical Analysis, Cambridge Univ. Press, 2003.
L.N. Trefethen & D. Bau III, Numerical Linear Algebra, SIAM, 1997.
C.F. Van Loan, K.-Y. Daysy Fan, Insight Through Computing, SIAM, 2010.
H. B. Wilson, L. H. Turcotte, D. Halpern, Advanced Mathematics and Mechanics Applications using MATLAB, Chapman and Hall, 2003, 3rd ed.
K. Yosida, Functional Analysis, Springer-Verlag, 1980, 6–th ed.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Lyche, T., Merrien, JL. (2014). Linear Least Squares Methods. In: Exercises in Computational Mathematics with MATLAB. Problem Books in Mathematics. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-43511-3_10
Download citation
DOI: https://doi.org/10.1007/978-3-662-43511-3_10
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-43510-6
Online ISBN: 978-3-662-43511-3
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)