The Extended Feature LMS Algorithm: Exploiting Hidden Sparsity for Systems with Unknown Spectrum

Abstract

The feature least-mean-square (F-LMS) algorithm has already been introduced to exploit hidden sparsity in lowpass and highpass systems. In this paper, by proposing the extended F-LMS (EF-LMS) algorithm, we boosted the F-LMS algorithm to exploit hidden sparsity in more general systems, those which are neither lowpass nor highpass. To this end, by means of the so-called feature matrix, we reveal the hidden sparsity in coefficients and utilize the \(l_1\)-norm to exploit the exposed sparsity. As a result, the EF-LMS algorithm will improve the convergence rate and the steady-state mean-squared error (MSE) as compared to the traditional least-mean-square algorithm. Moreover, in this work, we analyze the convergence behavior of the coefficient vector and the steady-state MSE performance of the EF-LMS algorithm. Through synthetic and real-world experiments, it has been seen that the EF-LMS algorithm can improve the convergence rate and the steady-state MSE whenever the hidden sparsity is revealed.

Introduction

Adaptive filtering is an active field in signal processing and has found applications in several areas such as communications [30], robotics [27], sonar and radar [19], biomedical engineering [5, 23], vehicles [16, 38], and noise control [1, 24], to name just a few. Among various adaptive filtering algorithms proposed in the last six decades, the LMS algorithm [32] has been used enormously due to its low computational resources, simple implementation, and inexpensive hardware requirements [8, 13, 28]. The LMS algorithm and its variants are mostly utilized in real problems such as system identification [4, 14], signal prediction [34], echo cancellation [11, 25], adaptive array beamforming [26], and active noise control [1, 24].

Related works

In practice, we may have some a priori information about the coefficients of an unknown system to be estimated; however, the LMS algorithm estimates the coefficients of a linear system without exploiting any a priori knowledge in the unknown system. One of the most critical prior information about the unknown system is the sparsity level of the system. Therefore, in the last two decades, various machine learning and adaptive filtering algorithms have been proposed in order to exploit sparsity in systems to improve their convergence rate or their mean-squared error (MSE) [2, 17, 18, 22, 31, 35,36,37]. All of these algorithms deal with plain sparsity; however, there are many examples where sparsity is not explicit and visible in system’s coefficients, and some appropriate technique is needed to reveal and exploit hidden sparsity. Indeed, sometimes the sparsity is in the linear combination of neighboring coefficients of unknown systems. This kind of sparsity is called hidden sparsity [9].

A system with hidden sparsity does not necessarily contain coefficients equal or close to zero, but the linear combination of its coefficients are equal or close to zero. Therefore: first, the hidden sparsity must be revealed using a linear transformation; then, we can exploit the revealed sparsity using conventional approaches. Recently, the feature least-mean-square (F-LMS) algorithms have been proposed to exploit hidden sparsity in systems with lowpass, highpass, and bandpass spectrum contents [9, 10, 33]. These algorithms utilize the feature matrix to expose the hidden sparsity in lowpass, highpass, and bandpass systems; however, they are not capable of exploiting hidden sparsity in systems with unknown spectrum.

Innovative contributions

In this paper, we propose the extended F-LMS (EF-LMS) algorithm by extending the applicability of the feature matrix so that it can detect similarities between adjacent coefficients of a system, independent of the system spectrum content. The EF-LMS extends the use of the F-LMS algorithm not only for lowpass, highpass, and bandpass systems but also for systems with other types of spectral correlation. To design the EF-LMS algorithm, we compare the values and the signs of neighboring coefficients. If two adjacent coefficients have similar values (like in lowpass systems), then they are considered similar, and the EF-LMS algorithm forces them to be equal. Also, if two adjacent coefficients have similar absolute values with different signs (like in highpass systems), then they are assumed similar, and the EF-LMS algorithm compels them to have identical values but with different signs.

The conventional F-LMS algorithms need mandatory prior knowledge about the spectrum content of unknown systems [6, 9, 10, 33]. However, the EF-LMS algorithm does not require a priori information about the spectrum content of unknown systems, which permits this algorithm to be applied in real-life scenarios. The EF-LMS algorithm has the best performance among the tested algorithms in experimental results. In particular, in real-life problems, it has a remarkable superior performance when compared to other tested algorithms due to its independence from the spectrum content of the unknown system to be identified.

Organization

This paper is organized as follows. In Sect. 2, we review the objective function of the F-LMS algorithm and propose the EF-LMS algorithm. Also, in this section, we address the computational complexity of the EF-LMS algorithm as compared to the LMS algorithm. Section 3 discusses the convergence behavior of the coefficient vector and the steady-state MSE performance of the EF-LMS algorithm. Section 4 provides some synthetic and real-world numerical examples utilizing the EF-LMS algorithm. For the synthetic data, the EF-LMS algorithm is applied to system identification problems. Furthermore, this algorithm is used to identify room impulse responses (RIR) in a real-world application. Finally, the conclusions are drawn in Sect. 5.

Notations and acronyms: Scalars are represented by lowercase letters. Vectors (matrices) are denoted by lowercase (uppercase) boldface letters. The ith entry of a given vector \({\mathbf {z}}\) is denoted by \(z_i\). For a given iteration k, the weight vector and the input vector are denoted by \({\mathbf {w}}(k),{\mathbf {x}}(k)\in {\mathbb {R}}^{N+1}\), respectively, where N is the adaptive filter order. The a priori error signal at the kth iteration is defined as \(e(k)\triangleq d(k)-{\mathbf {w}}^\text {T}(k){\mathbf {x}}(k)\), where \(d(k)\in {\mathbb {R}}\) is the desired signal, and the superscript \((\cdot )^\text {T}\) stands for vector transposition. The \(l_1\)-norm and the \(l_2\)-norm of a vector \({\mathbf {w}}\in {\mathbb {R}}^{N+1}\) are given by \(\Vert {\mathbf {w}}\Vert _1=\sum _{i=0}^N|w_i|\) and \(\Vert {\mathbf {w}}\Vert ^2={\mathbf {w}}^\text {T}{\mathbf {w}}=\sum _{i=0}^Nw_i^2\), respectively. The sign function is denoted by \(\mathrm{sgn}\), and it is defined as

$$\begin{aligned} \mathrm{sgn}(x)=\left\{ \begin{array}{ll}1,&{}\quad \text {if~}x>0,\\ 0,&{}\quad \text {if~}x=0,\\ -1&{}\quad \text {if~}x<0.\end{array}\right. \end{aligned}$$
(1)

Also, the employed acronyms in this paper are as follows:

\(\bullet \) LMS: least-mean-square \(\bullet \) F-LMS: feature LMS
\(\bullet \) EF-LMS: extended F-LMS \(\bullet \) MSE: mean-squared error
\(\bullet \) RIR: room impulse response \(\bullet \) EMSE: excess MSE
\(\bullet \) SNR: signal-to-noise ratio \(\bullet \) FIR: finite impulse response

The Extended F-LMS algorithm

The F-LMS is an LMS-type algorithm designed to exploit the hidden sparsity in unknown systems with either low or high frequency contents [9]. This algorithm utilizes the following objective function to update the system coefficients,

$$\begin{aligned} \xi _{\text {F-LMS}} (k) = \underbrace{ \frac{1}{2}|e(k)|^2 }_{\text {standard LMS term}} + \underbrace{\alpha \Vert {\mathbf {F}}(k) {\mathbf {w}}(k) \Vert _1}_{\text {feature-inducing term}}, \end{aligned}$$
(2)

where \(\alpha \in {\mathbb {R}}_+\) is the weight given to the \(l_1\)-norm, and \({\mathbf {F}}(k)\) is the feature matrix. The role of this matrix is to transform the adaptive filter \({\mathbf {w}}(k)\) (which does not need to be sparse in its original domain) to a different domain where the transformed coefficients are sparse. In other words, the feature matrix \({\mathbf {F}}(k)\) maps the non-sparse vector \({\mathbf {w}}(k)\) to the sparse vector \({\mathbf {F}}(k){\mathbf {w}}(k)\). Then the exposed sparsity in the transformed domain (i.e., \({\mathbf {F}}(k){\mathbf {w}}(k)\)) is exploited using the \(l_1\)-norm. Note that some other candidates, acting as sparsity-promoting penalty functions, can be used to replace the \(l_1\)-norm [3, 12, 18, 29, 35, 36].

The fundamental advantages of the F-LMS algorithm are attained thanks to the feature matrix, which plays a primary role in generating the appropriate F-LMS algorithm. However, the original F-LMS algorithm, with the feature matrix introduced in [9], requires strong a priori information about the unknown system. Therefore, it cannot exploit suitably the hidden sparsity eventually present in general systems, those which are neither lowpass nor highpass. In this section, we intend to propose a novel version of the F-LMS algorithm in such a way that it is capable of exploiting hidden sparsity in a broader class of unknown systems.

In general, there might exist hidden relations between adjacent coefficients of unknown systems. In this case, few coefficients may have abrupt and irrelevant changes as compared to their neighbors. If we exploit the relation between adjacent coefficients appropriately, the hidden sparsity in the unknown system would be revealed and would benefit the learning process. Indeed, if vector \({\mathbf {w}}_*\) denotes the unknown system, we should construct a feature matrix \({\mathbf {F}}\) such that \({\mathbf {F}}{\mathbf {w}}_*\) generates a sparse vector; then we can exploit this sparsity with the help of sparsity-promoting penalty functions. In order to avoid the requirement of a priori knowledge about the spectrum content of the unknown system, we should define a feature matrix such that it seeks the relations between adjacent coefficients independently of the spectrum content of the system.

A strong characteristic of unknown coefficients is their energy or absolute value, and the feature matrix can focus on recognizing the similarity between the absolute values of adjacent coefficients. However, two adjacent coefficients with similar absolute values can have the same or opposite signs. Therefore, at a given iteration k, we introduce the feature matrix \({\mathbf {F}}(k)\in {\mathbb {R}}^{N\times (N+1)}\) as

$$\begin{aligned} {\mathbf {F}}(k)\triangleq \left[ \begin{array}{ccccc}r^+_{\epsilon ,0}(k)&{}\quad r^-_{\epsilon ,0}(k)&{}\quad 0&{}\quad \cdots &{}\quad 0\\ 0&{}\quad r^+_{\epsilon ,1}(k)&{}\quad r^-_{\epsilon ,1}(k)&{}\quad 0&{}\quad \vdots \\ \vdots &{}\quad \ddots &{}\quad \ddots &{}\quad \ddots &{}\quad 0\\ 0&{}\quad \cdots &{}\quad 0&{}\quad r^+_{\epsilon ,N-1}(k)&{}\quad r^-_{\epsilon ,N-1}(k)\end{array}\right] , \end{aligned}$$
(3)

where \(r^+_{\epsilon ,i}(k)\triangleq g_\epsilon (v_i(k))+g_\epsilon (z_i(k))\), \(r^-_{\epsilon ,i}(k)\triangleq g_\epsilon (v_i(k))-g_\epsilon (z_i(k))\), \(v_i(k)\triangleq w_i(k)+w_{i+1}(k)\), and \(z_i(k)\triangleq w_i(k)-w_{i+1}(k)\), for \(i=0,\ldots ,N-1\). Moreover, the function \(g_\epsilon :{\mathbb {R}}\rightarrow {\mathbb {R}}\) is defined byFootnote 1

$$\begin{aligned} g_\epsilon (x)\triangleq \left\{ \begin{array}{ll}1, &{}\quad \text {if }|x|<\epsilon , \\ 0,&{}\quad \text {if }|x|\ge \epsilon ,\end{array}\right. \end{aligned}$$
(4)

\(\epsilon \in {\mathbb {R}}_+\) being a small positive constant utilized to detect sharp transitions between the absolute values of adjacent coefficients.

To understand \({\mathbf {F}}(k)\) better, for a given iteration k, we discuss different possible scenarios in the following:

  • When two adjacent coefficients of \({\mathbf {w}}(k)\) with indexes i and \(i+1\) have close values (i.e., their difference is less than \(\epsilon \)), then \(|z_i(k)|<\epsilon \) and \(|v_i(k)|>\epsilon \); it results in \(g_\epsilon (z_i(k))=1\) and \(g_\epsilon (v_i(k))=0\). Thus, the ith row of \({\mathbf {F}}(k)\) contains 1 and -1 on its ith and \((i+1)\)th columns, respectively, and the ith element of \({\mathbf {F}}(k){\mathbf {w}}(k)\), \(z_i(k)\), is close to zero since its absolute value is less than \(\epsilon \).

  • When two adjacent coefficients of \({\mathbf {w}}(k)\) with indexes i and \(i+1\) have close absolute values with different signs (i.e., their sum is less than \(\epsilon \)), then \(|z_i(k)|>\epsilon \) and \(|v_i(k)|<\epsilon \); it leads to \(g_\epsilon (z_i(k))=0\) and \(g_\epsilon (v_i(k))=1\). Hence, the ith row of \({\mathbf {F}}(k)\) contains 1 on its ith and \((i+1)\)th columns, and the ith element of \({\mathbf {F}}(k){\mathbf {w}}(k)\), \(v_i(k)\), is close to zero since its absolute value is less than \(\epsilon \).

  • If there is no similarity between two adjacent coefficients of \({\mathbf {w}}(k)\) with indexes i and \(i+1\) (i.e., neither their difference nor their sum is less than \(\epsilon \)), then both \(|z_i(k)|\) and \(|v_i(k)|\) are greater than \(\epsilon \); it concludes \(g_\epsilon (z_i(k))=g_\epsilon (v_i(k))=0\). Therefore, the ith row of \({\mathbf {F}}(k)\) is identical to the null vector, and the ith element of \({\mathbf {F}}(k){\mathbf {w}}(k)\) is null.

  • When two adjacent coefficients of \({\mathbf {w}}(k)\) with indexes i and \(i+1\) have similarity in two aspects, i.e., their difference and their sum have absolute values less than \(\epsilon \), then we can easily conclude that \(|w_i(k)|\) and \(|w_{i+1}(k)|\) are less than \(\epsilon \). In this case, \(g_\epsilon (z_i(k))=g_\epsilon (v_i(k))=1\); thus, the ith row of \({\mathbf {F}}(k)\) contains the digit 2 on its ith column and zero on other columns. Since \(|w_i(k)|<\epsilon \), the ith element of \({\mathbf {F}}(k){\mathbf {w}}(k)\) is less than \(2\epsilon \); i.e., for small \(\epsilon \), it is close to zero.

We can observe that, in all possible scenarios for the adjacent coefficients of \({\mathbf {w}}(k)\), the elements of the vector \({\mathbf {F}}(k){\mathbf {w}}(k)\) are less than \(2\epsilon \). Therefore, for small enough \(\epsilon >0\), \({\mathbf {F}}(k){\mathbf {w}}(k)\) is a sparse vector. Hence, we may now introduce the Extended F-LMS (EF-LMS) algorithm by considering the objective function in (2) with the feature matrix in (3). Using the stochastic gradient descent method for the objective function in (2), the recursion rule of the EF-LMS algorithm can be characterized by

$$\begin{aligned} {\mathbf {w}}(k+1)={\mathbf {w}}(k)+\mu e(k){\mathbf {x}}(k) -\mu \alpha {\mathbf {p}}(k), \end{aligned}$$
(5)

where \(\mu \in {\mathbb {R}}_+\) is the step-size parameter and should be adopted small enough to guarantee convergence [8]. Moreover, \({\mathbf {p}}(k)\in {\mathbb {R}}^{N+1}\) is the gradient of \(\Vert {\mathbf {F}}(k){\mathbf {w}}(k)\Vert _1\) with respect to \({\mathbf {w}}(k)\), when \({\mathbf {F}}(k)\) is given by (3). In this case, the expression of \({\mathbf {p}}(k)\) is described by (6), and it is worthwhile to mention that the computation of \({\mathbf {p}}(k)\) is inexpensive since it does not involve multiplication or division.

$$\begin{aligned} p_i(k)=\left\{ \begin{array}{ll} g_\epsilon (z_0(k))\mathrm{sgn}(z_0(k))+g_\epsilon (v_0(k))\mathrm{sgn}(v_0(k)), &{}\quad \mathrm{if }\, i=0, \\ -g_\epsilon (z_{i-1}(k))\mathrm{sgn}(z_{i-1}(k))+g_\epsilon (z_{i}(k))\mathrm{sgn}(z_{i}(k)) &{}\quad \\ +g_\epsilon (v_{i-1}(k))\mathrm{sgn}(v_{i-1}(k))+g_\epsilon (v_{i}(k))\mathrm{sgn}(v_{i}(k)) &{}\quad \mathrm{if}\, i=1,\ldots ,N-1,\\ -g_\epsilon (z_{N-1}(k))\mathrm{sgn}(z_{N-1}(k)) &{}\quad \\ +g_\epsilon (v_{N-1}(k))\mathrm{sgn}(v_{N-1}(k)), &{}\quad \mathrm{if~} i=N.\end{array}\right. \end{aligned}$$
(6)

Finally, the EF-LMS algorithm is summarized in Table 1. When there is a similarity between two adjacent coefficients of \({\mathbf {w}}(k)\), the EF-LMS algorithm exploits the hidden sparsity and enforces these coefficients to have similar absolute values. However, when there is no similarity between adjacent coefficients, the corresponding row of \({\mathbf {F}}(k)\) is identical to a null vector, and the EF-LMS algorithm does not force these coefficients to have a similarity; the algorithm learns these coefficients similarly as the LMS algorithm does.

Table 1 The Extended Feature LMS algorithm

Remark 1

The value of \(\epsilon \) should be small enough to detect a reasonable similarity between neighboring coefficients and to result in a high sparsity level for \({\mathbf {F}}(k){\mathbf {w}}(k)\). In real-life examples, empirically, we concluded that \(0.001\le \epsilon \le 0.005\) could be an appropriate candidate.

Remark 2

The computational complexity of the EF-LMS algorithm is a little higher than that of the LMS algorithm. In fact, for a given iteration, the EF-LMS and the LMS algorithms need a similar number of multiplications since the multiplications in Eq. (6) are trivial. The only exception is when adjacent coefficients have similarity in two aspects, i.e., the absolute values of their difference and their sum are less than \(\epsilon \), and it means that the system has plain sparsity. In this case, there are many other techniques in literature to exploit plain sparsity [2, 18, 22, 35, 36], and in this work our target is to exploit hidden sparsity. Hence, in systems containing hidden sparsity, the similarity in two aspects between adjacent coefficients is assumed unusual, and the number of required multiplication in the EF-LMS algorithm is similar to or a little more than that of the LMS algorithm.

Moreover, as compared to the LMS algorithm, the EF-LMS algorithm requires at most 6N more addition/subtraction per iteration. However, in practice, the difference of the required addition/subtraction operations between the EF-LMS and the LMS algorithms is definitely smaller than 6N per iteration. Also, at each iteration, the EF-LMS algorithm needs 6N comparison operations to implement Eq. (6).

Some Properties of the EF-LMS algorithm

In this section, we analyze the convergence behavior of the coefficient vector and the steady-state mean-squared error performance of the EF-LMS algorithm.

Convergence Behavior of the Coefficient Vector

Suppose that the coefficient vector of an unknown FIR filter is denoted by \({\mathbf {w}}_*\). Let us identify this unknown system by an adaptive filter of the same order using the EF-LMS algorithm. Also, assume that a measurement zero-mean white noise n(k) with variance \(\sigma _n^2\), statistically independent of the input signal x(k), is added to the output of the filter. Moreover, let us denote by \({\widetilde{{\mathbf {w}}}}(k)\) the difference between the adaptive filter coefficients \({\mathbf {w}}(k)\) and the unknown system \({\mathbf {w}}_*\); i.e., \({\widetilde{{\mathbf {w}}}}(k)\triangleq {\mathbf {w}}_*-{\mathbf {w}}(k)\). Therefore, we have

$$\begin{aligned} e(k)=d(k)-{\mathbf {w}}^\text {T}(k){\mathbf {x}}(k)={\mathbf {w}}_*^\text {T}{\mathbf {x}}(k)+n(k)-{\mathbf {w}}^\text {T}(k){\mathbf {x}}(k). \end{aligned}$$
(7)

Subtracting both sides of the update rule in (5) from \({\mathbf {w}}_*\) and utilizing (7), we obtain

$$\begin{aligned} {\widetilde{{\mathbf {w}}}}(k+1)=&{\widetilde{{\mathbf {w}}}}(k)-\mu e(k){\mathbf {x}}(k)+\mu \alpha {\mathbf {p}}(k)\nonumber \\ =&{\widetilde{{\mathbf {w}}}}(k)-\mu {\mathbf {x}}(k)[{\mathbf {x}}^\text {T}(k){\widetilde{{\mathbf {w}}}}(k)+n(k)]+\mu \alpha {\mathbf {p}}(k)\nonumber \\ =&[{\mathbf {I}}_{N+1}-\mu {\mathbf {x}}(k){\mathbf {x}}^\text {T}(k)]{\widetilde{{\mathbf {w}}}}(k)-\mu {\mathbf {x}}(k)n(k)+\mu \alpha {\mathbf {p}}(k), \end{aligned}$$
(8)

where \({\mathbf {I}}_{N+1}\) is the \((N+1)\times (N+1)\) identity matrix.

Assume that n(k), \({\mathbf {x}}(k)\), and \({\widetilde{{\mathbf {w}}}}(k)\) are statistically independent, thus by taking the expected value of both sides of the above equation, we obtain

$$\begin{aligned} E[{\widetilde{{\mathbf {w}}}}(k+1)]=[{\mathbf {I}}_{N+1}-\mu {\mathbf {R}}]E[{\widetilde{{\mathbf {w}}}}(k)]+\mu \alpha E[{\mathbf {p}}(k)], \end{aligned}$$
(9)

where \({\mathbf {R}}\triangleq E[{\mathbf {x}}(k){\mathbf {x}}^T(k)]\) denotes the autocorrelation matrix. From Eq. (6), we observe that the elements of \({\mathbf {p}}(k)\) are the sum of at most four sign functions. It implies that

$$\begin{aligned} -4\mu \alpha \mathbf{1}\preceq \mu \alpha E[{\mathbf {p}}(k)] \preceq 4\mu \alpha \mathbf{1}, \end{aligned}$$
(10)

where \(\mathbf{1}=[1~1~\ldots ~1]^\text {T}\), and \(\preceq \) stands for the componentwise inequality. Therefore, \(\mu \alpha E[{\mathbf {p}}(k)]\) is bounded and, by taking \(0<\mu <\frac{2}{\lambda _\text {max}}\) [7, 8], \(E[{\widetilde{{\mathbf {w}}}}(k+1)]\) will converge, \(\lambda _\text {max}\) being the largest eigenvalue of \({\mathbf {R}}\). Hence, by choosing \(0<\mu <\frac{2}{\lambda _\text {max}}\) and using Eq. (9), we have

$$\begin{aligned} \mu {\mathbf {R}}E[{\widetilde{{\mathbf {w}}}}(\infty )]=\mu \alpha E[{\mathbf {p}}(\infty )]. \end{aligned}$$
(11)

Assuming \({\mathbf {R}}\) is invertible, we obtain

$$\begin{aligned} E[{\widetilde{{\mathbf {w}}}}(\infty )]=\alpha {\mathbf {R}}^{-1}E[{\mathbf {p}}(\infty )]. \end{aligned}$$
(12)

This equation states that, by adopting a proper step-size parameter, the EF-LMS algorithm will never diverge. Finally, the validity of Eq. (12) is verified by numerical simulations in Sect. 4.1.

Steady-State MSE Performance

To analyze the steady-state MSE performance of the EF-LMS algorithm, let us subtract both sides of the update equation in (5) from \({\mathbf {w}}_*\), then we get

$$\begin{aligned} {\widetilde{{\mathbf {w}}}}(k+1)={\widetilde{{\mathbf {w}}}}(k)-\mu e(k){\mathbf {x}}(k)+\mu \alpha {\mathbf {p}}(k). \end{aligned}$$
(13)

From Eq. (7), we have

$$\begin{aligned} e(k)={\widetilde{{\mathbf {w}}}}^\text {T}(k){\mathbf {x}}(k)+n(k)={\widetilde{e}}(k)+n(k), \end{aligned}$$
(14)

where \({\widetilde{e}}(k)={\widetilde{{\mathbf {w}}}}^\text {T}(k){\mathbf {x}}(k)\) is the noiseless error. Thus, by multiplying each side of (13) to its transpose, we attain

$$\begin{aligned} \Vert {\widetilde{{\mathbf {w}}}}(k+1)\Vert ^2=&\Vert {\widetilde{{\mathbf {w}}}}(k)\Vert ^2-\mu e(k){\widetilde{e}}(k)+\mu \alpha {\widetilde{{\mathbf {w}}}}^\text {T}(k){\mathbf {p}}(k)\nonumber \\&-\mu e(k){\widetilde{e}}(k)+\mu ^2e^2(k){\mathbf {x}}^\text {T}(k){\mathbf {x}}(k)-\mu ^2\alpha e(k){\mathbf {x}}^\text {T}(k){\mathbf {p}}(k)\nonumber \\&+\mu \alpha {\mathbf {p}}^\text {T}(k){\widetilde{{\mathbf {w}}}}(k)-\mu ^2\alpha e(k){\mathbf {p}}^\text {T}(k){\mathbf {x}}(k)+\mu ^2\alpha ^2{\mathbf {p}}^\text {T}(k){\mathbf {p}}(k). \end{aligned}$$
(15)

Since \(\mu \) and \(\alpha \) are small constants, we can neglect the terms multiplied by \(\mu ^2\alpha \) and \(\mu ^2\alpha ^2\). Hence, using (14), we obtain

$$\begin{aligned} \Vert {\widetilde{{\mathbf {w}}}}(k+1)\Vert ^2=&\Vert {\widetilde{{\mathbf {w}}}}(k)\Vert ^2-2\mu {\widetilde{e}}^2(k)-2\mu {\widetilde{e}}(k)n(k)\nonumber \\&+2\mu \alpha {\widetilde{{\mathbf {w}}}}^\text {T}(k){\mathbf {p}}(k)+\mu ^2({\widetilde{e}}(k)+n(k))^2\Vert {\mathbf {x}}(k)\Vert ^2. \end{aligned}$$
(16)

Moreover, in the steady-state, we can assume that n(k), \({\widetilde{e}}(k)\), and x(k) are statistically independent. Thus, by taking the expected value of the above equation, we get

$$\begin{aligned} E[\Vert {\widetilde{{\mathbf {w}}}}(k+1)\Vert ^2]=&E[\Vert {\widetilde{{\mathbf {w}}}}(k)\Vert ^2]-2\mu E[{\widetilde{e}}^2(k)]+2\mu \alpha E[{\widetilde{{\mathbf {w}}}}^\text {T}(k){\mathbf {p}}(k)]\nonumber \\&+\mu ^2E[{\widetilde{e}}^2(k)]\mathrm{tr}({\mathbf {R}})+\mu ^2\sigma _n^2\mathrm{tr}({\mathbf {R}}), \end{aligned}$$
(17)

where \(\mathrm{tr}({\mathbf {R}})\) denotes the trace of \({\mathbf {R}}\). When the EF-LMS algorithm converges to the steady-state, we can assume that \(E[\Vert {\widetilde{{\mathbf {w}}}}(k+1)\Vert ^2]=E[\Vert {\widetilde{{\mathbf {w}}}}(k)\Vert ^2]\). Therefore, we get

$$\begin{aligned} 2\mu E[{\widetilde{e}}^2(k)]-\mu ^2E[{\widetilde{e}}^2(k)]\mathrm{tr}({\mathbf {R}})=\mu ^2\sigma _n^2\mathrm{tr}({\mathbf {R}})+2\mu \alpha E[{\widetilde{{\mathbf {w}}}}^\text {T}(k){\mathbf {p}}(k)]. \end{aligned}$$
(18)

Hence, the steady-state excess MSE (EMSE) of the EF-LMS algorithm is given by

$$\begin{aligned} E[{\widetilde{e}}^2(k)]=\frac{\mu \sigma _n^2\mathrm{tr}({\mathbf {R}})+2\alpha E[{\widetilde{{\mathbf {w}}}}^\text {T}(k){\mathbf {p}}(k)]}{2-\mu \mathrm{tr}({\mathbf {R}})}. \end{aligned}$$
(19)

Experimental Results

In this section, we provide some experimental results to verify the performance of the EF-LMS algorithm in comparison with a few other algorithms. To this end, we present simulation experiments in Sect. 4.1 and real-life experiments in Sect. 4.2 and 4.3.

Simulation Experiments

In this subsection, we utilize the LMS, the F-LMS [9], and the proposed EF-LMS algorithms in system identification problems. For the F-LMS algorithm, we require the a priori information about the spectrum content of the unknown system; thus, we use the F-LMS algorithm for lowpass systems [9]. The input signal is a zero-mean white Gaussian noise with unit variance, and all algorithms are initialized with the null vector. The parameter \(\alpha \) is set to 0.05. For each scenario, the step-size parameter will be informed later in the figures. The signal-to-noise ratio (SNR) is set to 20 dB, and the MSE learning curves are computed by averaging the outcomes of 200 independent trials.

The first unknown system, \({\mathbf {w}}_*\), is of order 39, i.e., it contains 40 coefficients, and its impulse response is presented in Fig. 1a. Indeed, it is comprised of eight blocks with equal lengths, where adjacent coefficients inside four blocks have similar values but in the other four blocks, the adjacent coefficients have similar absolute values with different signs. The parameter \(\epsilon \) in the EF-LMS algorithm is chosen as 0.03. Indeed, in this synthetic example, since we know the system in advance, we adopted the best value for \(\epsilon \). Figure 1b shows the MSE learning curves of the tested algorithms. As can be seen, choosing \(\mu =0.03\), the EF-LMS algorithm has the highest convergence rate and the lowest misadjustment, followed by the LMS and the F-LMS algorithms. We can observe that the EF-LMS algorithm outperforms the F-LMS algorithm since, differently from the F-LMS algorithm, it can exploit the similarity between all adjacent coefficients independent of the spectrum content of the system. Indeed, the F-LMS algorithm has a poor performance since it uses a time-invariant feature matrix which enforces adjacent coefficients with no similarity to have identical values. Furthermore, when the LMS algorithm has adopted a small step-size, 0.01, to attain the same MSE as that of the EF-LMS algorithm, we can observe that its convergence speed degrades significantly. It is worth mentioning that, in Fig. 1b, the learning curves start decreasing for \(k>N\) since, in the system identification scenarios, the input signal vector is composed of a delay line with a single input signal, i.e., \({\mathbf {x}}(k)=[x(k)~x(k-1)~\ldots ~x(k-N)]^\text {T}\) and, for \(-N\le i\le -1\), \(x(i)=0\).

Fig. 1
figure1

a The impulse response of \({\mathbf {w}}_*\); b the MSE learning curves considering \({\mathbf {w}}_*\)

The second unknown system, \({\mathbf {w}}_*'\), is a bandpass system of order 199 (i.e., 200 coefficients) with central frequency at \(\frac{\pi }{4}\), the lower and the upper cut-off frequencies at \(\frac{\pi }{4}-0.2\) and \(\frac{\pi }{4}+0.2\), respectively, the lower and the upper transition frequencies at \(\frac{\pi }{4}-0.3\) and \(\frac{\pi }{4}+0.3\), respectively. The impulse response of \({\mathbf {w}}_*'\) is depicted in Fig. 2a. In this scenario, for the F-LMS algorithm, we adopt the feature matrix for lowpass systems. However, the EF-LMS algorithm utilizes the feature matrix proposed in Eq. (3), and it does not require the a propri information about the spectrum content of the unknown system. The step-size parameter is chosen as 0.007 for all tested algorithms, and \(\epsilon =0.005\) in the EF-LMS algorithm. In this example, the MSE learning curves are the average of outcomes of 1000 independent ensembles. Figure 2b illustrates the MSE learning curves of the LMS, the F-LMS, and the EF-LMS algorithms. As can be seen, the EF-LMS algorithm has the highest convergence rate and the lowest misadjustment, followed by the LMS and the F-LMS algorithms. Again, it is worth mentioning that the learning curves start decreasing for \(k>N\) since the input signal vector is adopted as \({\mathbf {x}}(k)=[x(k)~x(k-1)~\ldots ~x(k-N)]^\text {T}\) and, for \(-N\le i\le -1\), \(x(i)=0\).

Fig. 2
figure2

a The impulse response of \({\mathbf {w}}_*'\); b the MSE learning curves considering \({\mathbf {w}}_*'\)

Moreover, Fig. 3 presents the MSE learning curves of the LMS, the F-LMS, and the EF-LMS algorithms for a time-varying system. In the first 1500 iterations, the bandpass system \({\mathbf {w}}'_*\) is considered as the unknown system and, at the iteration 1501, the unknown system is changed to a system of order 199 containing four lowpass blocks and four highpass blocks of the same length. For all tested algorithms, the step-size parameter is selected as 0.007, and \(\epsilon =0.005\) in the EF-LMS algorithm. As can be observed, the EF-LMS algorithm can perfectly track the abrupt change in the system, and it attained the lowest MSE among the tested algorithms.

Fig. 3
figure3

The MSE learning curves considering a time-varying system

Fig. 4
figure4

The MSE learning curves considering \({\mathbf {w}}_*''\)

The fourth unknown system, \({\mathbf {w}}_*''\), is of order 39, and its coefficients are drawn independently from a zero-mean Gaussian distribution with unit variance. In this case, there is no relation between adjacent coefficients. The step-size parameter is adopted as 0.03 for all employed algorithms and \(\epsilon =0.005\) for the EF-LMS algorithm. Figure 4 shows the MSE learning curves of the LMS, the F-LMS, and the EF-LMS algorithms considering \({\mathbf {w}}_*''\). As can be seen, when there is no relation between adjacent coefficients, the performance of the LMS and the EF-LMS algorithms are incredibly similar; however, the F-LMS algorithm has significantly higher steady-state MSE. This example shows that when there is no relation between the adjacent coefficients of a system, by adopting a proper \(\epsilon \), the EF-LMS algorithm will match with the LMS algorithm. However, when there is some relation between the adjacent coefficients, as in the previous examples, the EF-LMS algorithm outperforms the LMS algorithm.

Fig. 5
figure5

The values of \(\Vert E[{\mathbf {w}}(\infty )-{\mathbf {w}}_*]\Vert \) and \(\Vert \alpha {\mathbf {R}}^{-1}E[{\mathbf {p}}(\infty )]\Vert \) for different step-sizes, when \({\mathbf {w}}_*\) is considered as the unknown system

Fig. 6
figure6

The values of \(w_i(k)\), for \(i=0,5,10,15,20,25,30,35\), when \({\mathbf {w}}_*\) is considered as the unknown system

Furthermore, to corroborate the validity of the proposed analysis in Sect. 3.1, the Euclidean norm of both sides of Eq. (12) are presented in Fig. 5, when \(\epsilon =0.03\) and \({\mathbf {w}}_*\), shown in Fig. 1a, is adopted as the unknown system. To obtain the results in Fig. 5, for different step-size parameters \(0.003\le \mu \le 0.035\), the EF-LMS algorithm has been implemented for 6000 iterations, and the averages of \({\mathbf {w}}(k)-{\mathbf {w}}_*\) and \(\alpha {\mathbf {R}}^{-1}{\mathbf {p}}(k)\) over 1000 independent runs have been computed to produce the ensemble-averages. Then the time-averages of the last 4000 iterations have been calculated. Finally, for \(0.003\le \mu \le 0.035\), the Euclidean norms of these values are illustrated in Fig. 5. As can be seen, this figure substantiates the validity of Eq. (12). Moreover, we can observe that, for different step-sizes, the blue curve (circles), \(\Vert E[{\mathbf {w}}(\infty )-{\mathbf {w}}_*]\Vert \), is extremely close to zero; it means that the EF-LMS algorithm has converged to the optimum solution \({\mathbf {w}}_*\). For instance, when \(\mu =0.03\), the values of \(w_i(k)\), for \(i=0,5,10,15,20,25,30,35\), are presented in Fig. 6. We can see that all these coefficients have converged to the optimum coefficients after 600 iterations.

Also, Fig. 7 shows the theoretical and the experimental steady-state EMSE of the EF-LMS algorithm, for \(0.003\le \mu \le 0.035\), when \(\epsilon =0.03\) and \({\mathbf {w}}_*\) is chosen as the unknown system. The theoretical steady-state EMSE is computed by Eq. (19). To acquire the experimental EMSE, for \(0.003\le \mu \le 0.035\), we have executed the EF-LMS algorithm for 6000 iterations, and we have calculated the average of \({\widetilde{e}}^2(k)\) over 1000 independent trials to generate the ensemble-averages. Finally, the time-average of the last 4000 iterations has been computed. As can be observed in Fig. 7, there is an excellent match between theoretical and experimental results.

Fig. 7
figure7

The experimental and the theoretical steady-state EMSE of the EF-LMS algorithm for different step-sizes, when \({\mathbf {w}}_*\) is considered as the unknown system

Room Impulse Response Identification

As a real-life experiment, we have used the LMS and the EF-LMS algorithms to identify the measured room impulse response (RIR) for three microphones in a room receiving the sound generated by a loudspeaker. The microphones are B&K condenser microphones type 4165, and the loudspeaker is Yamaha MSP3. This experiment has been carried out in a room with dimensions and shape as depicted in Fig. 8, where we can observe the positions of the loudspeaker and of the microphones. Microphones were located: (1) in the middle of the room, (2) on one side of the room next to the wall, and (3) in the corner of the room. After been estimated offline, the RIR was utilized as the unknown system in the system identification problem using the LMS and the EF-LMS algorithms. The sampling frequency is set to 8 kHz. The input signal is a zero-mean white Gaussian noise with unit variance, and the measurement noise has zero-mean Gaussian distribution with variance 0.01. In this experiment, the value of \(\alpha \) is adopted as 0.02, and \(\epsilon =0.005\) for the EF-LMS algorithm. The step-size parameter \(\mu \) for these algorithms is informed in Fig. 10. Both algorithms are initialized with the null vector. For microphones 1 and 2, the order of the adaptive filter is 3800, while for microphone 3, it is 4000. The learning curves are the average of outcomes of 100 independent runs.

Fig. 8
figure8

The room and the positions of the loudspeaker and the microphones are illustrated in this figure

Fig. 9
figure9

Estimated room impulse responses: a microphone 1; b microphone 2; c microphone 3

Fig. 10
figure10

Learning curves: a microphone 1; b microphone 2; c microphone 3

The impulse responses for microphones 1, 2, and 3 are illustrated in Fig. 9a–c, respectively. Figure 10a–c illustrate the MSE learning curves of the LMS and the EF-LMS algorithms when they have been applied to identify the RIR by microphones 1, 2, and 3, respectively. As can be seen, for all microphones, the EF-LMS algorithm has lower misadjustment and higher convergence speed as compared to the LMS algorithm. Thus, this experiment highlights the benefits provided by the EF-LMS algorithm for a particular scenario when the system to be identified does not have any particular shape (lowpass or highpass, for instance). Moreover, the EF-LMS algorithm did not require significant computational resources in comparison with the LMS algorithm. As an example, when identifying the RIR of order \(N=3800\) with microphone 1, the EF-LMS algorithm has executed 222 multiplication operations and 9509 (\(\ll 6N\)) addition/subtraction operations more than the LMS algorithm per iteration. Furthermore, the EF-LMS algorithm has implemented \(6N=22{,}800\) comparison operations per iteration.

Underwater Channel Estimation

In this scenario, a channel impulse response for a shallow water receiver is employed in a system identification problem. The details of the measured channel impulse response are available in [15]. In a system identification scenario, we tried to estimate the impulse response of an underwater communication channel by the LMS and the EF-LMS algorithms. Four different input signals are adopted: (i) a zero-mean white Gaussian noise (WGN) with unit variance, (ii) a first-order moving average (MA) process generated by \(x(k)=m(k)+0.75m(k-1)\), where m(k) is a zero-mean WGN with unit variance; (iii) a first-order autoregressive (AR) process produced by \(x(k)=0.75x(k-1)+m(k)\); (iv) a first-order autoregressive moving average (ARMA) process generated by \(x(k)=0.75x(k-1)+0.75m(k-1)+m(k)\). Moreover, all input signals are normalized so that they have unit variance and zero mean. The measurement noise signal has a zero-mean white Gaussian distribution with variance 0.1; i.e., the SNR was set to 10 dB. In both algorithms, the adaptive filter order is 2880, \({\mathbf {w}}(0)\) is the null vector, and the step-size parameter is chosen as 0.0005. In the EF-LMS algorithm, the values of \(\alpha \) and \(\epsilon \) were set to 0.02 and 0.005, respectively. Moreover, the learning curves are obtained by averaging the outcomes of 100 independent trials.

Figure 11 depicts the MSE learning curves of the LMS and the EF-LMS algorithms when they were employed to identify the underwater channel impulse response. Indeed, the MSE learning curves of these algorithms are illustrated in Fig.11a–d when the input signals are adopted as WGN, MA, AR, and ARMA, respectively. As can be observed, in Figure 11a–c, the EF-LMS algorithm outperforms the LMS algorithm, reaching a significantly lower steady-state MSE and a slightly higher convergence rate. Also, even for the case of the ARMA input signal, as observed in Fig. 11d, the EF-LMS algorithm performs somewhat better than the LMS algorithm. Thus, for all adopted input signals, the EF-LMS algorithm is capable of exposing and exploiting the sparsity hidden in the underwater channel impulse response without any a priori knowledge about the spectrum content of the system.

Fig. 11
figure11

The MSE learning curves considering the underwater channel estimation scenario using: a WGN input signal; b MA input signal; c AR input signal; d ARMA input signal

Conclusion

In this paper, we have proposed the EF-LMS algorithm in order to exploit the hidden sparsity in a broader class of unknown systems. For this purpose, we have introduced a proper feature matrix so that, by multiplying the system coefficients by the feature matrix, we obtain a sparse vector. Once the hidden sparsity is revealed, it is exploited using the \(l_1\)-norm. Indeed, at each iteration, the difference and the sum of each two adjacent coefficients are computed: if the magnitudes of the obtained values are less than a predetermined value \(\epsilon >0\), the algorithm considers a similarity between these coefficients and exploits this property in the learning process. Moreover, we have analyzed the convergence behavior of the coefficient vector of the EF-LMS algorithm and have introduced an upper bound for the step-size parameter. The synthetic numerical results and the real-life simulations have highlighted the advantages provided by the EF-LMS algorithm.

The EF-LMS algorithm requires a little higher computational burden as compared to the LMS algorithm; however, it can attain lower misadjustment and higher convergence rate for a wider class of systems than for those where the F-LMS is applicable. In comparison with the F-LMS algorithm, the EF-LMS does not need a priori information about the spectrum content of systems to exploit hidden sparsity whereas, in using F-LMS algorithms, this prior knowledge is indispensable. Also, note that the parameter \(\epsilon \) in the EF-LMS algorithm is a key constant that should be chosen carefully to avoid low performance. Thus, we recommend underestimating this parameter to prevent bad selection. However, in future works, data-driven approaches can be utilized to learn this parameter from data. Moreover, as a potential future research direction, hidden sparsity can be exploited in some state-of-the-art artificial intelligence applications different from classical problems. As an example, the idea of hidden sparsity can be extended to nonlinear and Hammerstein adaptive filters [20, 21].

Notes

  1. 1.

    This function was inspired by the discard function proposed in [36].

References

  1. 1.

    M.S. Aslam, M.A.Z. Raja, A new adaptive strategy to improve online secondary path modeling in active noise control systems using fractional signal processing approach. Signal Process. 107, 433–443 (2015)

    Article  Google Scholar 

  2. 2.

    J. Benesty, S.L. Gay, An improved PNLMS algorithm, in Proceedings IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May, Dallas, USA (2002), pp. 1881–1884

  3. 3.

    Z.A. Bhotto, A. Antoniou, A family of shrinkage adaptive-filtering algorithms. IEEE Trans. Signal Process. 61, 1689–1697 (2013)

    MathSciNet  Article  Google Scholar 

  4. 4.

    N.I. Chaudhary, S. Zubair, M.S. Aslam et al., Design of momentum fractional LMS for Hammerstein nonlinear system identification with application to electrically stimulated muscle model. Eur. Phys. J. Plus 134, 407 (2019)

    Article  Google Scholar 

  5. 5.

    N.I. Chaudhary, S. Zubair, M.A.Z. Raja et al., Normalized fractional adaptive methods for nonlinear control autoregressive systems. Appl. Math. Model. 66, 457–471 (2019)

    MathSciNet  Article  Google Scholar 

  6. 6.

    G.S. Chaves, M.V.S. Lima, H. Yazdanpanah, P.S.R. Diniz, T.N. Ferreira, A simple sparsity-aware feature LMS algorithm, in Proceedings 27th European Signal Processing Conference (EUSIPCO) (A Coruna, Spain, Sept., 2019), pp. 1–5

  7. 7.

    Y. Chen, Y. Gu, A.O. Hero, Sparse LMS for system identification, in Proceedings IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (Taipei, Taiwan, Apr., 2009), pp. 3125–3128

  8. 8.

    P.S.R. Diniz, Adaptive Filtering: Algorithms and Practical Implementation, 4th edn. (Springer, New York, 2013)

    Google Scholar 

  9. 9.

    P.S.R. Diniz, H. Yazdanpanah, M.V.S. Lima, Feature LMS algorithms, in Proceedings IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), April, Calgary, Alberta, Canada (2018), pp. 4144–4148

  10. 10.

    P.S.R. Diniz, H. Yazdanpanah, M.V.S. Lima, Feature LMS algorithm for bandpass system models, in Proceedings 27th European Signal Processing Conference (EUSIPCO) (A Coruna, Spain, Sept., 2019), pp. 1–5

  11. 11.

    B. Farhang-Boroujeny, Fast LMS/Newton algorithms based on autoregressive modeling and their application to acoustic echo cancellation. IEEE Trans. Signal Process. 45, 1987–2000 (1997)

    Article  Google Scholar 

  12. 12.

    Y. Gu, J. Jin, S. Mei, \(l_0\) norm constraint LMS algorithm for sparse system identification. IEEE Signal Process. Lett. 16, 774–777 (2009)

    Article  Google Scholar 

  13. 13.

    S. Haykin, Adaptive Filter Theory, 4th edn. (Prentice Hall, Englewood Cliffs, NJ, 2002)

    Google Scholar 

  14. 14.

    S. He, Y. Lin, Cauchy distribution function-penalized LMS for sparse system identification. Circuits Syst. Signal Process. 38, 470–480 (2019)

    MathSciNet  Article  Google Scholar 

  15. 15.

    J. Huang, R. Diamant, Channel impulse responses from Mar. 2019 long range experiment (Mediterranean Sea), IEEE Dataport (2019). https://doi.org/10.21227/nzgr-ds72.

  16. 16.

    K. Jiang, H. Zhang, H.R. Karimi, J. Lin, L. Song, Simultaneous input and state estimation for integrated motor-transmission systems in a controller area network environment via an adaptive unscented Kalman filter, IEEE Trans. Syst. Man Cybern Syst. (2018). https://doi.org/10.1109/TSMC.2018.2795340

  17. 17.

    Y. Li, C. Zhang, S. Wang, Low-complexity non-uniform penalized affine projection algorithm for sparse system identification. Circuits Syst. Signal Process. 35, 1611–1624 (2016)

    Article  Google Scholar 

  18. 18.

    M.V.S. Lima, T.N. Ferreira, W.A. Martins, P.S.R. Diniz, Sparsity-aware data-selective adaptive filters. IEEE Trans. Signal Process. 62, 4557–4572 (2014)

    MathSciNet  Article  Google Scholar 

  19. 19.

    A. Mandal, R. Mishra, Digital equalization for cancellation of noise-like interferences in adaptive spatial filtering. Circuits Syst. Signal Process. 36, 675–702 (2017)

    Article  Google Scholar 

  20. 20.

    A. Mehmood, N.I. Chaudhary, A. Zameer et al., Novel computing paradigms for parameter estimation in Hammerstein controlled auto regressive auto regressive moving average systems. Appl. Soft Comput. 80, 263–284 (2019)

    Article  Google Scholar 

  21. 21.

    A. Mehmood, A. Zameer, N.I. Chaudhary et al., Backtracking search heuristics for identification of electrical muscle stimulation models using Hammerstein structure. Appl. Soft Comput. 84, 105705 (2019)

    Article  Google Scholar 

  22. 22.

    R. Meng, R.C. de Lamare, V.H. Nascimento, Sparsity-aware affine projection adaptive algorithms for system identification, in Proceedings Sensor Signal Processing for Defence (SSPD) (London, UK, Sept., 2011), pp. 1–5

  23. 23.

    L. Murali, D. Chitra, T. Manigandan, B. Sharanya, An efficient adaptive filter architecture for improving the seizure detection in EEG signal. Circuits Syst Signal Process 35, 2914–2931 (2016)

    Article  Google Scholar 

  24. 24.

    T. Padhi, M. Chandra, A. Kar, M.N.S. Swamy, A new hybrid active noise control system with convex combination of time and frequency domain filtered-x LMS algorithms. Circuits Syst. Signal Process. 37, 3275–3294 (2018)

    MathSciNet  Article  Google Scholar 

  25. 25.

    K. Pu, J. Zhang, L. Min, A signal decorrelation PNLMS algorithm for double-talk acoustic echo cancellation. Circuits Syst. Signal Process. 35, 669–684 (2016)

    Article  Google Scholar 

  26. 26.

    M.A.Z. Raja, R. Akhtar, N.I. Chaudhary et al., A new computing paradigm for the optimization of parameters in adaptive beamforming using fractional processing. Eur. Phys. J. Plus 134, 275 (2019)

    Article  Google Scholar 

  27. 27.

    G. Reina, A. Vargas, K. Nagatani, K. Yoshida, Adaptive Kalman filtering for GPS-based mobile robot localization, in Proceedings IEEE International Workshop on Safety, Security and Rescue Robotics (Rome, Italy, Sept., 2007), pp. 1–6

  28. 28.

    A.H. Sayed, Adaptive Filters (Wiley-IEEE, New York, USA, 2008)

    Google Scholar 

  29. 29.

    K. Shi, P. Shi, Adaptive sparse Volterra system identification with \(l_0\)-norm penalty. Signal Process. 91, 2432–2436 (2011)

    Article  Google Scholar 

  30. 30.

    A. Tarighat, A.H. Sayed, Least mean-phase adaptive filters with application to communications systems. IEEE Signal Process. Lett. 11, 220–223 (2004)

    Article  Google Scholar 

  31. 31.

    S. Werner, J.A. Apolinário Jr., P.S.R. Diniz, Set-membership proportionate affine projection algorithms. EURASIP J. Audio Speech Music Process (2007). https://doi.org/10.1155/2007/34242

  32. 32.

    B. Widrow, M.E. Hoff, Adaptive switching circuits. IRE WESCOM Conv. Rec. 4, 96–104 (1960)

    Google Scholar 

  33. 33.

    H. Yazdanpanah, J.A. Apolinário Jr., P.S.R. Diniz, M.V.S. Lima, \(l_0\)-norm feature LMS algorithms, in Proceeding 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Nov., Anaheim, CA, USA, (2018), pp. 311–315

  34. 34.

    H. Yazdanpanah, P.S.R. Diniz, New trinion and quaternion set-membership affine projection algorithms. IEEE Trans. Circuits Syst. II Express Briefs 64, 216–220 (2017)

  35. 35.

    H. Yazdanpanah, P.S.R. Diniz, Recursive Least-Squares algorithms for sparse system modeling, in Proceedings IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (New Orleans, LA, USA, Mar., 2017), pp. 3879–3883

  36. 36.

    H. Yazdanpanah, P.S.R. Diniz, M.V.S. Lima, A simple set-membership affine projection algorithm for sparse system modeling, in Proceedings 24th European Signal Processing Conference (EUSIPCO) (Budapest, Hungary, Aug., 2016), pp. 1798–1802

  37. 37.

    Y. Yu, H. Zhao, B. Chen, Sparseness-controlled proportionate affine projection sign algorithms for acoustic echo cancellation. Circuits Syst. Signal Process. 34, 3933–3948 (2015)

    MathSciNet  Article  Google Scholar 

  38. 38.

    H. Zhang, J. Wang, Active steering actuator fault detection for an automatically-steered electric ground vehicle. IEEE Trans. Veh. Technol. 66, 3685–3702 (2017)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the São Paulo Research Foundation (FAPESP) Grants #2015/22308-2, #2019/06280-1 and the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – Finance Codes 23038.018065/2018-17 and 88881.371305/2019-01.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Hamed Yazdanpanah.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yazdanpanah, H., Apolinário, J.A. The Extended Feature LMS Algorithm: Exploiting Hidden Sparsity for Systems with Unknown Spectrum. Circuits Syst Signal Process 40, 174–192 (2021). https://doi.org/10.1007/s00034-020-01461-3

Download citation

Keywords

  • Adaptive filtering
  • LMS algorithm
  • Feature
  • Sparsity
  • \(l_1\)-norm
  • Computational complexity