Abstract
The method of steepest descent described and developed in the previous chapter forms the mathematical basis for many current adaptive signal processing algorithms. However, steepest descent was developed for iteratively solving the normal equations (2.3.5) for the optimal w * N . For an actual signal processing application, this would be equivalent to requiring that the time series d(n) and x(n) be stationary and, additionally, that their second-order statistics be known. Knowledge of the second-order statistics in (2.3.5) was conveyed by the autocorrelation matrix and cross-correlation vector. However, in practical system implementations, these correlation values can only be estimated from available data, and this is often a source of computational delay, or error, or both. This chapter develops an alternative to the method of steepest descent called the least mean squares (LMS) algorithm, which will then be applied to problems in which the second-order statistics of the signal are unknown. Due to its simplicity, the LMS algorithm is perhaps the most widely used adaptive algorithm in currently implemented systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
B. Widrow, “Adaptive Filters,” inAspects of Network and System Theory, N. de Claris and R.E. Kaiman, eds., Holt, Rinehart, and Winston, New York, 1971.
B. Widrow, et al., “Adaptive Noise Cancelling: Principles and Applications,”Proceedings of the IEEE, vol. 63, pp. 1962 - 1976, December 1975
B. Widrow, et al., “Adaptive Antenna Systems,”Proceedings of the IEEE, vol. 55, pp. 2143–2159, December 1967
R.A. Monzingo and T.W. Miller,Introduction to Adaptive Arrays, Wiley- Interscience, New York, 1980.
L.J. Griffiths, “Rapid Measurement of Digital Instantaneous Frequency,”IEEE Trans, on Acous., Speech and Signal Processing, vol. ASSP-22, pp. 207–222, April 1975.
J. Treichler, “Transient and Convergent Behavior of the ALE,”IEEE Trans, on Acous., Speech and Signal Processing, vol. ASSP-27, pp. 53–63, February 1979.
R.W. Lucky, “Techniques for Adaptive Equalization of Digital Communications Systems,” Bell Sys. Tech. J., vol. 45, no. 2, 1966.
M.M. Sondhi, “An Adaptive Echo Canceller,” Bell Sys. Tech. J., vol. 46, no. 3, 1967.
S.T. Alexander and S.A. Rajala, “Image Compression Results Using the LMS Adaptive Algorithm,” IEEE Trans, on Acous., Speech, and Signal Processing, vol. ASSP-33, pp. 712–715, June 1985.
J. Kim and L. Davisson, “Adaptive Linear Estimation for Stationary M-Dependent Processes,” IEEE Trans, on Information Theory, vol. IT-21, pp. 23–31, January 1975.
T. Daniell, “Adaptive Estimation with Mutually Correlated Training Samples,” IEEE Trans. Sys. Sei. Cybernetics, vol. SSC-6, pp. 12–19, January 1970.
J. Treichler, “Response of the Adaptive Line Enhancer to Chirped and Doppler Shifted Sinusoids,” IEEE Trans, on Acous., Speech and Signal Processing, vol. ASSP-28, pp. 343–348, June 1980.
B. Widrow, J. McCool, and M. Ball, “The Complex LMS Algorithm,” Proceedings of the IEEE, vol. 63, pp. 719–720, April 1975.
N. Bershad, et al., “Tracking Characteristics of the LMS Adaptive Line Enhancer Response to a Linear Chirp Signal in Noise,” IEEE Trans, on Acous. Speech, and Signal Processing, vol. ASSP-28, pp. 504–516, October 1980.
B. Widrow and J. McCool, “A Comparison of Adaptive Algorithms Based on the Methods of Steepest Descent and Random Search,” IEEE Trans. Antennas and Propagat., vol. AP-24, pp. 615–637, September 1976.
L.L. Horowitz and K.D. Senne, “Performance Advantage for Complex LMS for Controlling Narrowband Adaptive Arrays,” IEEE Trans, on Acous., Speech, and Signal Processing, vol. ASSP-29, pp. 722–736, June 1981.
B. Fisher and N.J. Bershad, “The Complex LMS Adaptive Algorithm—Transient Weight Mean and Covariance with Applications to the ALE,” IEEE Trans, on Acous., Speech and Signal Processing, vol. ASSP-31, pp. 34–45, February 1983.
S.T. Alexander and S.A. Rajala, “Optimal Gain Derivation for the LMS Algorithm Using a Visual Fidelity Criteria,” IEEE Trans, on Acous. Speech, and Signal Processing, vol. ASSP-32, pp. 432–434, April 1984.
A.J. Berni, “Weight Jitter Phenomena in Adaptive Control Loops,”IEEE Trans. Aerosp. and Elec. Sys., vol. AES-14, pp. 355–361, July 1977.
N.J. Bershad and L.Z. Qu, “On the Joint Characteristic Function of the Complex Scalar LMS Weight,”IEEE Trans, on Acous., Speech and Signal Processing, vol. ASSP-32, pp. 1166–1175, December 1984.
R.A. Iltis and L.B. Milstein, “An Approximate Statistical Analysis of the Widrow LMS Algorithm with Application to Narrow-Band Interference Rejection,”IEEE Trans, on Communications, vol. COM-33, pp. 121–130, February 1985.
B. Widrow and S.D. Stearns,Adaptive Signal Processing, Prentice-Hall, Englewood Cliffs, NJ, 1985.
C.F.N. Cowan and P.M. Grant,Adaptive Filters, Prentice-Hall, Englewood Cliffs, NJ, 1985.
M.L. Honig and D.G. Messerschmitt,Adaptive Filters, Kluwer Academic Publishers, Hingham, MA, 1984.
S. Haykin,Introduction to Adaptive Filters, Macmillan, New York, 1984.
H.J. Trussell and J.D. Wang, “Cancellation of Harmonic Noise in Distribution Line Communications,” IEEE Trans, on Power Apparatus and Systems, vol. PAS- 104, pp. 3338–3344, December 1985.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1986 Springer-Verlag New York Inc.
About this chapter
Cite this chapter
Alexander, S.T. (1986). The Least Mean Squares (LMS) Algorithm. In: Adaptive Signal Processing. Texts and Monographs in Computer Science. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-4978-8_5
Download citation
DOI: https://doi.org/10.1007/978-1-4612-4978-8_5
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4612-9382-8
Online ISBN: 978-1-4612-4978-8
eBook Packages: Springer Book Archive