Abstract
In the two previous chapters, attention was paid on the algorithms for the determination or estimation of filters parameters with a methodology that provides knowledge of the processes statistics or their a priori calculated estimation on an appropriate window signal length. In particular, with regard to the choice of the cost function (CF) to be minimized J(w), the attention has been paid both to the solution methods of the Wiener–Hopf normal equations, which provide a stochastic optimization MMSE solution, and to the form of Yule–Walker that assumed a deterministic (or stochastic approximated) approach, by a least squares error (LSE) solution.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Note that the subscript n represents an iteration index not necessarily temporal.
- 2.
Subtracting w opt from both members of w n = w n−1 + μ(g−Rw n−1) = w n−1 + μ R(w opt−w n−1), we get w n − w opt = w n−1 − w opt + μ R(w opt − w −1) ⇒ u n = (I − μ R)u n−1.
- 3.
For independence, it holds that E{u[n]v[n]} = E{u[n]} ⋅ E{v[n]}.
- 4.
The reader will note that, with the direct-averaging approximation, the assumption of independence is not strictly necessary, since it takes into account implicitly.
- 5.
- 6.
In LMS the vector u n is an RV, for which (5.65) J n = J min + u H n Ru n , related to the SDA, in this case must be written as J n = J min + E{u H n − 1 Ru n−1}.
- 7.
Note that identical result can be reached considering a CF defined as
w ∴ min δ(w H n−1 w n−1) s.t. |e[n]|2 = |d[n] − w H n − 1 x|2,
with Lagrangian L(w,λ) = δ w H n − 1 w n−1 + λ|e[n]|2 and considering the descent in the stochastic gradient of the Lagrangian surface \( {\mathbf{w}}_n={\mathbf{w}}_{n-1}-\mu {\scriptscriptstyle \frac{1}{2}}{\nabla}_{\mathbf{w}}L\left(\mathbf{w},\boldsymbol{\uplambda} \right) \).
References
Levenberg K (1944) A method for the solution of certain problems in least squares. Quart Appl Math 2:164–168
Marquardt D (1963) An algorithm for least squares estimation on nonlinear parameters. Siam J Appl Math 11:431–441
Fletcher R (1986) Practical methods of optimization. Wiley, New York, NY. ISBN 0471278289
Al-Naffouri TY, Sayed AH, Nascimento VH (2003) Energy conservation in adaptive filtering. In: Barner E, Arce G (eds) Nonlinear signal and image processing: theory, methods, and applications. CRC Press, Boca Raton, FL, pp 1–35
Sayed AH (2003) Fundamentals of adaptive filtering. IEEE Wiley - Interscience, New York, NY
Yousef NR, Sayed AH (2001) A unified approach to the steady-state and tracking analyses of adaptive filters. IEEE Trans Signal Proc 49(2)
Al-Naffouri TY, Sayed AH (2003) Transient analysis of adaptive filters with error nonlinearities. IEEE Trans Signal Proc 51(3):653–663
Al-Naffouri TY, Sayed AH (2003) Transient analysis of data-normalized adaptive filters. IEEE Trans Signal Proc 51(3):639–652
Haykin S (1996) Adaptive filter theory, Third Editionth edn. Prentice Hall, Upper Saddle River, NJ
Widrow B, Hoff ME (1960) Adaptive switching circuits. IRE WESCON, Conv. Rec., pt. 4:96–104
Widrow B (1966) Adaptive filters I: fundamentals. Stanford Electron. Labs, Stanford, CA, SEL-66-126
Godara LC, Cantoni A (1986) Analysis of constrained LMS algorithm with application to adaptive beamforming using perturbation sequences. IEEE Trans Antennas Propagat AP-34(3):368–379
Kushner HJ (1984) Approximation and weak convergence methods for random processes, with applications to stochastic systems theory. MIT Press, Cambridge, MA. ISBN 0262110903, 9780262110907
Clarkson PM, White PR (1987) Simplified analysis of the LMS adaptive filter using a transfer function approximation. IEEE Trans Acoustics Speech Signal Proc ASSP-35(7):987–933
Duttweiler DL (2000) Proportionate normalized least-mean-squares adaptation in echo cancelers. IEEE Trans Speech Audio Proc 8:508–518
Benesty J, Gay SL (2002) An improved PNLMS algorithm. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '02:1881–1884
Huang Y, Benesty J, Chen J (2006) Acoustic MIMO signal processing. Springer Series on Signal and Communication Technology, ISBN 10 3-540-37630-5
Kamenetsky M, Widrow B (2004) A variable leaky lms adaptive algorithm. IEEE Thirty-Eighth Asilomar Conference on Signals, Systems and Computers 1:125–128
Mayyas K, Tyseer A (1997) Leaky LMS algorithm: MSE analysis for Gaussian data. IEEE Trans Signal Proc 45:927–934
Walach E, Widrow B (1984) The least mean fourth (LMF) adaptive algorithm and its family. IEEE Trans Inform Theor IT-30(2):215
Long G, Ling F, Proakis J (1989) The LMS with delayed coefficient adaptation. IEEE Trans Acoustics Speech Signal Proc 37:1397–1405
Long G, Ling F, Proakis J (1992) Corrections to the LMS with delayed coefficient adaptation. IEEE Trans Signal Proc 40:230–232
Rupp M, Frenzel R (1994) Analysis of LMS and NLMS algorithms with delayed coefficient update under the presence of spherically invariant processes. IEEE Trans Signal Proc 42:668–672
Tobias OJ, Bermudez JCM, Bershad NJ (2000) Stochastic analysis of the delayed LMS algorithm for a new model. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '00. Vol. 1, pp 404–407, 15-9 June 2000
Widrow B, Stearns SD (1985) Adaptive signal processing. Prentice Hall, Upper Saddle River, NJ
Morgan DR (1980) An analysis of multiple correlation cancellation loops with a filter in the auxiliary path. IEEE Trans Acoust Speech Signal Proc ASSP-28(4):454–467
Widrow B, Shur D, Shaffer S (1981) On adaptive inverse control. In 15th Asilomar Conference Circuits, Systems, and Components, 185–189
Elliott SJ, Stothers IM, Nelson PA (1987) A multiple error LMS algorithm and its application to the active control of sound and vibration. lEEE Trans Acoust Speech Signal Proc ASSP-35(10):1423–1434
Tobias OJ, Bermudez JCM, Bershad NJ (2000) Mean weight behavior of the filtered-X LMS algorithm. IEEE Trans Signal Proc 48:1061–1075
Snyder SD, Hansen CH (1994) Effect of transfer function estimation errors on the filtered-X LMS algorithm. IEEE Trans Signal Proc 42(4):950–953
Bjarnason E (1995) Analysis of the filtered-X LMS algorithm. IEEE Trans Speech Audio Proc 3:504–514
Douglas S, Pan W (1995) Exact expectation analysis of the LMS adaptive filter. IEEE Trans Signal Proc 43:2863–2871
Boucher CC, Elliott SJ, Nelson PA (1991) Effect of errors in the plant model on the performance of algorithms for adaptive feed forward control. Proc Inst Elect Eng F 138:313–319
Wan EA (1996) Adjoint LMS: an efficient alternative to the filtered-X LMS and multiple error LMS algorithms. Proc IEEE ICASSP-1996:1842–1845
Widrow B (1971) Adaptive filters, from aspect of networks and system theory. Kalman – De Claris (ed.) Holt, Rinehart and Winston
Widrow B et al (1975) Adaptive noise cancellation: principles and applications. Proc IEEE 63:1691–1717
Wiener N (1949) Extrapolation, interpolation and smoothing of stationary time series, with engineering applications. Wiley, New York, NY
Bode HW, Shannon CE (1950) A simplified derivation of linear least squares smoothing and prediction theory. Proc IRE 38:417–425
Kivinen J, Warmuth MK (1997) Exponential gradient versus gradient descent for linear prediction. Inform Comput 132:1–64
Widrow B, McCool J, Ball M (1975) The complex LMS algorithm. Proc IEEE 63(4):719–720
Feuer A, Weinstein E (1985) Convergence analysis of LMS filters with uncorrelated Gaussian data. IEEE Trans Acoustics Speech Signal Proc 33(1):222–230
Farhang-Boroujeny B (1998) Adaptive filters: theory and applications. Wiley, New York, NY
Manolakis DG, Ingle VK, Kogon SM (2000) Statistical and adaptive signal processing. McGraw-Hill, New York, NY
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Uncini, A. (2015). First-Order Adaptive Algorithms. In: Fundamentals of Adaptive Signal Processing. Signals and Communication Technology. Springer, Cham. https://doi.org/10.1007/978-3-319-02807-1_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-02807-1_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-02806-4
Online ISBN: 978-3-319-02807-1
eBook Packages: EngineeringEngineering (R0)