Skip to main content
SpringerLink
Log in

The Least-Mean-Square (LMS) Algorithm

  • Chapter
  • First Online:
Adaptive Filtering

Abstract

The least-mean-square (LMS) is a search algorithm in which simplification of the gradient vector computation is made possible by appropriately modifying the objective function [1, 2]. The review [3] explains the history behind the early proposal of the LMS algorithm, whereas [4] places into perspective the importance of this algorithm. The LMS algorithm, as well as others related to it, is widely used in various applications of adaptive filtering due to its computational simplicity [5,6,7,8,9]. The convergence characteristics of the LMS algorithm are examined in order to establish a range for the convergence factor that will guarantee stability.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 129.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Because it minimizes the mean of the squared error. It belongs to the class of stochastic-gradient algorithms.

  2. 2.

    This choice also guarantees the convergence of the MSE.

  3. 3.

    This is an abuse of language, by infinite precision we mean very long wordlength.

  4. 4.

    The missing factor 2 here originates from the term \(\frac{1}{2}\) in definition of the gradient that we opted to use in order to be coherent with most literature, in actual implementation the factor 2 of the real case is usually incorporated to the \(\mu \).

  5. 5.

    The M-ary QAM constellation points are represented in by , with \(\tilde{a}_{i} = \pm \tilde{d}, \pm 3 \tilde{d}, \ldots , \pm (\sqrt{M}-1) \tilde{d}\), and \(\tilde{b}_{i} = \pm \tilde{d}, \pm 3 \tilde{d}, \ldots , \pm (\sqrt{M}-1) \tilde{d}\). The parameter \(\tilde{d}\) is represents half of the distance between two points in the constellation.

References

  1. B. Widrow, M.E. Hoff, Adaptive switching circuits. WESCOM Conv. Rec. 4, 96–140 (1960)

    Google Scholar 

  2. B. Widrow, J.M. McCool, M.G. Larimore, C.R. Johnson Jr., Stationary and nonstationary learning characteristics of the LMS adaptive filters. Proc. IEEE 64, 1151–1162 (1976)

    Article  MathSciNet  Google Scholar 

  3. B. Widrow, D. Park, History of adaptive signal processing: Widrow’s group, in A Short History of Circuits and Systems, eds. by F. Maloberti, A.C. Davies (River Publishers, Delft, 2016)

    Google Scholar 

  4. P.S.R. Diniz, B. Widrow, History of adaptive filters, in A Short History of Circuits and Systems, eds. by F. Maloberti, A.C. Davies (River Publishers, Delft, 2016)

    Google Scholar 

  5. G. Ungerboeck, Theory on the speed of convergence in adaptive equalizers for digital communication. IBM J. Res. Dev. 16, 546–555 (1972)

    Article  Google Scholar 

  6. J.E. Mazo, On the independence theory of equalizer convergence. Bell Syst. Tech. J. 58, 963–993 (1979)

    Article  MathSciNet  Google Scholar 

  7. B. Widrow, S.D. Stearns, Adaptive Signal Processing (Prentice Hall, Englewood Cliffs, 1985)

    MATH  Google Scholar 

  8. S. Haykin, Adaptive Filter Theory, 4th edn. (Prentice Hall, Englewood Cliffs, 2002)

    MATH  Google Scholar 

  9. M.G. Bellanger, Adaptive Digital Filters and Signal Analysis, 2nd edn. (Marcel Dekker Inc, New York, 2001)

    Book  Google Scholar 

  10. D.C. Farden, Racking properties of adaptive signal processing algorithms. IEEE Trans. Acoust. Speech Signal Process. (ASSP) 29, 439–446 (1981)

    Google Scholar 

  11. B. Widrow, E. Walach, On the statistical efficiency of the LMS algorithm with nonstationary inputs. IEEE Trans. Inform. Theor. (IT) 30, 211–221 (1984)

    Google Scholar 

  12. O. Macchi, Optimization of adaptive identification for time varying filters. IEEE Trans. Automat. Contr. (AC) 31, 283–287 (1986)

    Google Scholar 

  13. A. Benveniste, Design of adaptive algorithms for the tracking of time varying systems. Int. J. Adapt. Contr. Signal Process. 1, 3–29 (1987)

    Article  Google Scholar 

  14. W.A. Gardner, Nonstationary learning characteristics of the LMS algorithm. IEEE Trans. Circ. Syst. (CAS) 34, 1199–1207 (1987)

    Google Scholar 

  15. A. Papoulis, Probability, Random Variables, and Stochastic Processes, 3rd edn. (McGraw Hill, New York, 1991)

    MATH  Google Scholar 

  16. F.J. Gantmacher, The Theory of Matrices, vol. 2 (Chelsea Publishing Company, New York, 1964)

    Google Scholar 

  17. G.H. Golub, C.F. Van Loan, Matrix Computations, 3rd edn. (John Hopkins University Press, Baltimore, 1996)

    MATH  Google Scholar 

  18. V. Solo, The limiting behavior of LMS. IEEE Trans. Acoust. Speech Signal Process. 37, 1909–1922 (1989)

    Article  MathSciNet  Google Scholar 

  19. N.J. Bershad, O.M. Macchi, Adaptive recovery of a chirped sinusoid in noise, part 2: performance of the LMS algorithm. IEEE Trans. Signal Process. 39, 595–602 (1991)

    Article  Google Scholar 

  20. D.H. Brandwood, A complex gradient operator and its application in adaptive array theory. IEE Proc. Parts F and G 130, 11–16 (1983)

    MathSciNet  Google Scholar 

  21. A. Hjørungnes, D. Gesbert, Complex-valued matrix differentiation: techniques and key results. IEEE Trans. Signal Process. 55, 2740–2746 (2007)

    Article  MathSciNet  Google Scholar 

  22. D.G. Manolakis, V.K. Ingle, S.M. Kogon, Statistical and Adaptive Signal Processing (McGraw Hill, New York, 2000)

    Google Scholar 

  23. O. Macchi, E. Eweda, Second-order convergence analysis of stochastic adaptive linear filter. IEEE Trans. Automat. Contr. (AC) 28, 76–85 (1983)

    Google Scholar 

  24. V.H. Nascimento, A.H. Sayed, On the learning mechanism of adaptive filters. IEEE Trans. Signal Process. 48, 1609–1625 (2000)

    Article  Google Scholar 

  25. S. Florian, A. Feuer, Performance analysis of the LMS algorithm with a tapped delay line (two-dimensional case). IEEE Trans. Acoust. Speech Signal Process. (ASSP) 34, 1542–1549 (1986)

    Google Scholar 

  26. O.L. Frost III, An algorithm for linearly constrained adaptive array processing. Proc. IEEE 60, 926–935 (1972)

    Article  Google Scholar 

  27. J.A. Apolinário Jr., S. Werner, T.I. Laakso, P.S.R. Diniz, Constrained normalized adaptive filtering for CDMA mobile communications, in Proceedings of 1998 EUSIPCO-European Signal Processing Conference, Rhodes, Greece (1998), pp. 2053–2056

    Google Scholar 

  28. J.A. Apolinário Jr., M.L.R. de Campos, C.P. Bernal O, The constrained conjugate-gradient algorithm. IEEE Signal Process. Lett. 7, 351–354 (2000)

    Google Scholar 

  29. M.L.R. de Campos, S. Werner, J.A. Apolinário Jr., Constrained adaptation algorithms employing Householder transformation. IEEE Trans. Signal Process. 50, 2187–2195 (2002)

    Article  MathSciNet  Google Scholar 

  30. A. Feuer, E. Weinstein, Convergence analysis of LMS filters with uncorrelated Gaussian data. IEEE Trans. Acoust. Speech Signal Process. (ASSP) 33, 222–230 (1985)

    Google Scholar 

  31. D.T. Slock, On the convergence behavior of the LMS and normalized LMS algorithms. IEEE Trans. Signal Process. 40, 2811–2825 (1993)

    Article  Google Scholar 

  32. W.A. Sethares, D.A. Lawrence, C.R. Johnson Jr., R.R. Bitmead, Parameter drift in LMS adaptive filters. IEEE Trans. Acoust. Speech Signal Process. (ASSP) 34, 868–878 (1986)

    Google Scholar 

  33. S.C. Douglas, Exact expectation analysis of the LMS adaptive filter. IEEE Trans. Signal Process. 43, 2863–2871 (1995)

    Article  Google Scholar 

  34. H.J. Butterweck, Iterative analysis of the state-space weight fluctuations in LMS-type adaptive filters. IEEE Trans. Signal Process. 47, 2558–2561 (1999)

    Article  Google Scholar 

  35. B. Hassibi, A.H. Sayed, T. Kailath, \(H^{\infty }\) optimality of the LMS algorithm. IEEE Trans. Signal Process. 44, 267–280 (1996)

    Article  Google Scholar 

  36. O.J. Tobias, J.C.M. Bermudez, N.J. Bershad, Mean weight behavior of the filtered-X LMS algorithm. IEEE Trans. Signal Process. 48, 1061–1075 (2000)

    Article  Google Scholar 

  37. V. Solo, The error variance of LMS with time varying weights. IEEE Trans. Signal Process. 40, 803–813 (1992)

    Article  Google Scholar 

  38. S.U. Qureshi, Adaptive equalization. Proc. IEEE 73, 1349–1387 (1985)

    Article  Google Scholar 

  39. M.L. Honig, Echo cancellation of voiceband data signals using recursive least squares and stochastic gradient algorithms. IEEE Trans. Comm. (COM) 33, 65–73 (1985)

    Google Scholar 

  40. V.H. Nascimento, J.C.M. Bermudez, Probability of divergence for the least-mean fourth algorithm. IEEE Trans. Signal Proces. 54, 1376–1385 (2006)

    Article  Google Scholar 

  41. M.T.M. Silva, V.H. Nascimento, Improving tracking capability of adaptive filters via convex combination. IEEE Trans. Signal Proces. 56, 3137–3149 (2008)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universidade Federal do Rio de Janeiro, Niterói, Rio de Janeiro, Brazil

    Paulo S. R. Diniz

Authors
  1. Paulo S. R. Diniz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paulo S. R. Diniz .

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Diniz, P.S.R. (2020). The Least-Mean-Square (LMS) Algorithm. In: Adaptive Filtering. Springer, Cham. https://doi.org/10.1007/978-3-030-29057-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-29057-3_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-29056-6

  • Online ISBN: 978-3-030-29057-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation