Skip to main content
SpringerLink
Log in
Book cover

Fundamentals of Adaptive Signal Processing pp 143–204Cite as

Least Squares Method

  • Chapter
  • First Online:

Part of the book series: Signals and Communication Technology ((SCT))

Abstract

This chapter introduces the deterministic counterpart of the statistical Wiener filter theory. The problems of adaptation are addressed in the case where the filter input signals are sequences generated by linear deterministic models without any assumption on their statistics.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   119.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    The method of least squares was introduced by the German mathematician Carl Friedrich Gauss in 1795, at age 18, and was used by him for the calculation of the orbit of the asteroid Ceres in 1821.

  2. 2.

    Note that the constrained optimization is a methodology used very often for the determination of the optimal solution in particular adaptive filtering problems. See Appendix B for deepening.

  3. 3.

    Note that this solution is equivalent to the δ-solution described in Sect. 4.3.1.2.

  4. 4.

    The term matching pursuit indicates, in general, a numerical method for selecting the best projection (also known as best matching) of multidimensional data in a over-complete basis.

  5. 5.

    A set of vectors x∈(ℝ,ℂ)N satisfies the Haar condition if every set of N vectors is linearly independent. In other words, each subset selection of N vectors, from a base for the space (ℝ,ℂ)N. A system of equations that satisfies the Haar condition is sometimes referred to as Tchebycheff system [21, 30].

References

  1. Kay SM (1993) Fundamental of statistical signal processing estimation theory. Prentice Hall, Englewood Cliffs, NJ

    Google Scholar 

  2. Kailath T (1974) A view of three decades of linear filtering theory. IEEE Trans Inform Theor IT20(2):146–181

    Article  MathSciNet  Google Scholar 

  3. Box GEP, Jenkins GM (1970) Time series analysis: forecasting and control. Holden-Day, San Francisco, CA

    MATH  Google Scholar 

  4. Bates DM, Watts DG (1988) Nonlinear regression analysis and its applications. Wiley, New York

    Book  MATH  Google Scholar 

  5. Landweber L (1951) An iteration formula for Fredholm integral equations of the first kind. Am J Math 73:615–624

    Article  MathSciNet  MATH  Google Scholar 

  6. Kaczmarz S (1937) Angenäherte Auflösung von Systemen linearer Gleichungen. Bulletin International de l’Académie Polonaise des Sciences et des Lettres Classe des Sciences Mathématiques et Naturelles Série A, Sciences Mathématiques 35:355–357

    Google Scholar 

  7. Manolakis DG, Ingle VK, Kogon SM (2000) Statistical and adaptive signal processing. McGraw-Hill, New York

    Google Scholar 

  8. Golub GH, Van Loan CF (1989) Matrix computation. John Hopkins University Press, Baltimore, MD. ISBN 0-80183772-3

    Google Scholar 

  9. Strang G (1988) Linear algebra and its applications, 3rd edn. Thomas Learning, Cambridge. ISBN:0-15-551005-3

    Google Scholar 

  10. Petersen KB, Pedersen MS, The matrix cookbook. http://matrixcookbook.com, Ver. February 16, 2008

  11. Noble B, Daniel JW (1988) Applied linear algebra. Prentice-Hall, Englewood Cliffs, NJ

    Google Scholar 

  12. Haykin S (1996) Adaptive filter theory, 3rd edn. Prentice Hall, Englewood Cliffs, NJ

    Google Scholar 

  13. Cadzow JA, Baseghi B, Hsu T (1983) Singular-value decomposition approach to time series modeling. IEEE Proc Commun Radar Signal Process 130(3):202–210

    Article  Google Scholar 

  14. van der Veen AJ, Deprettere EF, Swindlehurst AL (1993) Subspace-based signal analysis using singular value decomposition. Proc IEEE 81(9):1277–1308

    Article  Google Scholar 

  15. Cichocki A, Amari SI (2002) Adaptive blind signal and image processing. Wiley, New York. ISBN 0-471-60791-6

    Book  Google Scholar 

  16. Cichocki A, Unbehauen R (1994) Neural networks for optimization and signal processing. Wiley, New York

    Google Scholar 

  17. Van Huffel S, Vandewalle J (1991) The total least squares problems: computational aspects and analysis, vol 9, Frontiers in applied mathematics. SIAM, Philadelphia, PA

    Book  MATH  Google Scholar 

  18. Golub GH, Van Loan CF (1980) An analysis of the total least squares problem. SIAM J Matrix Anal Appl 17:883–893

    MATH  Google Scholar 

  19. Farhang-Boroujeny B (1998) Adaptive filters: theory and applications. Wiley, New York

    Google Scholar 

  20. Sayed AH (2003) Fundamentals of adaptive filtering. Wiley, Hoboken, NJ. ISBN 0-471-46126-1

    Google Scholar 

  21. Golub GH, Hansen PC, O’Leary DP (1999) Tikhonov regularization and total least squares. SIAM J Matrix Anal Appl 21:185–194

    Article  MathSciNet  MATH  Google Scholar 

  22. Pascual-Marquia RD, Michel CM, Lehmannb D (1994) Low resolution electromagnetic tomography: a new method for localizing electrical activity in the brain. Int J Psychophysiol 18(1):49–65

    Article  Google Scholar 

  23. Gorodnitsky IF, Rao BD (1997) Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Trans Signal Process 45(3):600–616

    Article  Google Scholar 

  24. Rao BD, Engan K, Cotter SF, Palmer J, Kreutz-Delgado K (2003) Subset selection in noise based on diversity measure minimization. IEEE Trans Signal Process 51(3):760–770

    Article  Google Scholar 

  25. Wipf DP, Rao BD (2007) An empirical bayesian strategy for solving the simultaneous sparse approximation problem. IEEE Trans Signal Process 55(7):3704–3716

    Article  MathSciNet  Google Scholar 

  26. Zdunek R, Cichocki A (2008) Improved M-FOCUSS algorithm with overlapping blocks for locally smooth sparse signals. IEEE Trans Signal Process 56(10):4752–4761

    Article  MathSciNet  Google Scholar 

  27. He Z, Cichocki A, Zdunek R, Xie S (2009) Improved FOCUSS method with conjugate gradient iterations. IEEE Trans Signal Process 57(1):399–404

    Article  MathSciNet  Google Scholar 

  28. Xu P, Tian Y, Chen H, Yao D (2007) L p norm iterative sparse solution for EEG source localization. IEEE Trans Biomed Eng 54(3):400–409

    Article  Google Scholar 

  29. Rao BD, Kreutz-Delgado K (1999) An affine scaling methodology for best basis selection. IEEE Trans Signal Process 47(1):187–200

    Article  MathSciNet  MATH  Google Scholar 

  30. Cheney EW (1999) Introduction to approximation theory, 2nd edn. American Mathematical Society, Providence, RI

    Google Scholar 

  31. Golub G, Pereyra V (2003) Separable nonlinear least squares: the variable projection method and its applications. Inverse Probl 19 R1. doi:10.1088/0266-5611/19/2/201

    Google Scholar 

  32. Cadzow JA (1990) Signal processing via least squares error modeling. IEEE ASSP Magazine, pp 12–31, October 1990

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. DIET, Sapienza University of Rome, Rome, Italy

    Aurelio Uncini

Authors
  1. Aurelio Uncini
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Uncini, A. (2015). Least Squares Method. In: Fundamentals of Adaptive Signal Processing. Signals and Communication Technology. Springer, Cham. https://doi.org/10.1007/978-3-319-02807-1_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-02807-1_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-02806-4

  • Online ISBN: 978-3-319-02807-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation