Two variants of the IIR spline adaptive filter for combating impulsive noise
- 219 Downloads
Abstract
It has been pointed out that the nonlinear spline adaptive filter (SAF) is appealing for modeling nonlinear systems with good performance and low computational burden. This paper proposes a normalized least M-estimate adaptive filtering algorithm based on infinite impulse respomse (IIR) spline adaptive filter (IIR-SAF-NLMM). By using a robust M-estimator as the cost function, the IIR-SAF-NLMM algorithm obtains robustness against non-Gaussian impulsive noise. In order to further improve the convergence rate, the set-membership framework is incorporated into the IIR-SAF-NLMM, leading to a new set-membership IIR-SAF-NLMM algorithm (IIR-SAF-SMNLMM). The proposed IIR-SAF-SMNLMM inherits the benefits of the set-membership framework and least-M estimate scheme and acquires the faster convergence rate and effective suppression of impulsive noise on the filter weight and control point adaptation. In addition, the computational burdens and convergence properties of the proposed algorithms are analyzed. Simulation results in the identification of the IIR-SAF nonlinear model show that the proposed algorithms offer the effectiveness in the absence of non-Gaussian impulsive noise and robustness in non-Gaussian impulsive noise environments.
Keywords
Nonlinear adaptive filter IIR spline adaptive filter Set-membership Least M-estimateAbbreviations
- APA
Affine projection algorithm
- AWGN
Additive white Gaussian noise
- IIR
Infinite impulse response
- LMS
Least mean square
- LN
Linear-nonlinear
- LNL
Linear-nonlinear-linear
- LTI
Linear time-invariant
- LUT
Look-up table
- MSE
Mean square error
- NL
Nonlinear-linear
- NLMM
Normalized least M-estimate
- NLMS
Normalized least mean square
- NLN
Nonlinear-linear-nonlinear
- NN
Neural networks
- SAF
Spline adaptive filter
- VAF
Volterra adaptive filters
1 Introduction
Due to their concise design and low complexity, the adaptive linear filters have gained wide attention in system modeling and identification [1, 2]. The adaptive linear filter is conventionally modeled as a finite impulse response (FIR) filter or an infinite impulse response (IIR) filter. Its tap weights are updated iteratively by using adaptive algorithms such as the least mean square (LMS) algorithm, normalized least mean square (NLMS) algorithm, and affine projection algorithm (APA). However, in the case of nonlinear system, linear models are inadequate and suffer from the performance losses due to the failure to model the nonlinearity. Hence, in order to model the nonlinearity, several adaptive nonlinear structures have been presented such as truncated Volterra adaptive filters (VAF) [3], neural networks (NNs) [4], block-oriented architecture [5], and spline adaptive filters (SAF) [6, 7, 8, 9]. Truncated VAF, originated from the Taylor series expansion, is one of the most used model for the nonlinearity. However, its implementation is limited because of a huge complexity requirement, in particular, for high-order Volterra models. To overcome the drawbacks of the truncated VAF, one of the most well-known structure is the block-oriented nonlinear architecture, which can be represented by the connections of linear time-invariant (LTI) models and memoryless nonlinear functions. There are several basic classes of the block-oriented nonlinear structure including the Wiener model [10], the Hammerstein model [11], and the variants originated from these two classes in accordance with different topologies (i.e., parallel, feedback, and cascade). Specifically, the Wiener model consists of a cascade of a linear LTI filter followed by a static nonlinear function which sometimes is deemed as linear-nonlinear (LN) model, and the Hammerstein model comprises a static nonlinear function connected behind a linear LTI filter which usually is considered as nonlinear-linear (NL) model. The cascade model, such as linear-nonlinear-linear (LNL) model or nonlinear-linear-nonlinear (NLN) model, has been proved to be more suitable for the generality of the model to be identified [12]. NNs are a flexible application for modeling nonlinearity, but it suffers from a large computational cost and difficulties in online adaptation.
Recently, combining the block-oriented architecture with the spline function, several novel adaptive nonlinear spline adaptive filters (SAFs) have been introduced such as Wiener spline filter, Hammerstein spline filter, cascade spline filter and IIR spline adaptive filter (IIR-SAF). These spline adaptive models can be implemented by different connections of the spline function and linear time-invariant (LTI) model. The nonlinearity in this kind of structure is modeled by the spline function, which can be represented by the adaptive look-up table (LUT) interpolated by a local low-order spline curve. The SAFs achieve improved performance in modeling the nonlinearity. Furthermore, in each iteration, only a portion of the control points is tuned depending on the order of the spline function and the nonlinear shape is slightly changed. Consequently, this local behavior of the spline function results in the considerable saving in the computation complexity.
Note that in all spline filters mentioned above, their update rules are based on the mean square error (MSE) criterion in additive white Gaussian noise (AWGN) environment which considers the cost function J=E[e^{2}(n)], where E[·] denotes the mathematical expectation and e(n) is the output error. However, in some cases of non-Gaussian noise such as underwater acoustic signal processing [13], radar signal processing [14], and communication systems [15], the SAFs may suffer from performance deterioration or failure to be robust against non-Gaussian noises. To address this problem, a least M-estimate scheme [16, 17] has been proposed by using the least M-estimator as the cost function which achieves the satisfactory performance when the input and desired signals are corrupted by non-Gaussian impulsive noises.
In this paper, extending the least M-estimate idea into the IIR-SAF, a normalized least M-estimate adaptive filtering algorithm based on IIR spline adaptive filter (IIR-SAF-NLMM) is proposed for nonlinear system identification. The update rule is based on the modified Huber M-estimate function, thus yielding a good effectiveness in suppressing non-Gaussian impulsive noises. To further improve the convergence performance of the IIR-SAF-NLMM, we incorporate the set-membership framework into the IIR-SAF-NLMM and propose a set-membership IIR-SAF-NLMM (IIR-SAF-SMNLMM) algorithm. It is derived by minimizing a new M-estimate-based cost function associated with a robust set-membership error bound. Due to the combination of the robust set-membership error bound and threshold parameter used to reject the outliers, the IIR-SAF-SMNLMM provides faster convergence rate and robustness against non-Gaussian impulsive noise compared with the conventional SAF algorithms.
The paper is organized as follows. The IIR-SAF structure is reviewed in Section 2. In Section 3, we derive the IIR-SAF-NLMM and IIR-SAF-SMNLMM algorithms. The computational complexity is given in Section 4, and convergence properties of the IIR-SAF-SMNLMM are analyzed in Section 5. Some simulation results are demonstrated in Section 6. Finally, Section 7 concludes the paper.
2 IIR-SAF structure
where w(n) is the weight vector of the IIR filter which is defined as w(n)=[b_{0}(n),b_{1}(n),⋯,b_{M−1}(n),a_{1}(n),⋯,a_{N}(n)]^{T}, b_{l}(n)(l=0,1,⋯,M−1), and a_{k}(n)(k=1,2, ⋯,N) denote the lth coefficient of the MA part and kth coefficient of the AR part in the IIR adaptive filter respectively. \(\bar {\mathbf {x}}(n)\,=\, \left [x(n),x(n - 1), \cdots, x(n - M + 1),s(n - 1), \cdots,s(n - N)\right ]^{T}\) is the input vector of the IIR filter.
where Q is the number of the control point, Δx is the uniform space between two adjacent control points, and ⌊·⌋ denotes the floor operator.
where μ_{w} and μ_{q} are the step-sizes in the linear network and nonlinear network, respectively; the small positive constant ε is used for avoiding zero division. The vector g_{n} is defined as g_{n}= [ ∂s(n)/∂b_{0}(n),⋯, ∂s(n)/∂b_{M−1}(n),∂s(n)/∂a_{1}(n),⋯,∂s(n)/∂a_{N}(n)]^{T}, and \(\dot {\mathbf {u}}_{n} = \left [3u_{n}^{2},2u_{n}^{},1,0\right ]^{T}\), e(n) is the output error which can be expressed as \(e (n) = d(n) - y(n) = d(n) - \mathbf {u}_{n}^{T} {\mathbf {Cq}}_{i,n}\), where d(n) is the desired signal which contains non-Gaussian impulsive noises.
3 Proposed IIR-SAF-NLMM and IIR-SAF-SMNLMM algorithms
3.1 IIR-SAF-NLMM algorithm
where λ_{0} is the forgetting factor close to but smaller than 1, a_{1}=1.483(1+5/(N_{w}−1)) is a finite correction factor, and N_{w} is the data window. med[·] denotes the median operator and C_{e}(n)=[e^{2}(n),e^{2}(n−1),⋯,e^{2}(n−N_{w}+1)].
Note that in [17], the threshold parameter ξ is evaluated with the assumption of Gaussian distribution of the output error. However, even in the case that e(n) is subject to other distribution, we also can compute the threshold value which is used to reject the impulse in output errors.
It can be seen in (12) and (14) that the output error is replaced by the score function ψ[ e(n)], resulting into the freezing on the update of the IIR weight vector and control point vector when the output error is larger than the threshold parameter. This way helps the IIR-SAF-SNLMM algorithm to suppress the adverse effect of the non-Gaussian impulsive noise.
3.2 IIR-SAF-SMNLMM algorithm
where Θ is the feasibility set in which all the tap-weight vectors are available for |d−w^{T}x|≤γ, and L denotes the linear filter length.
Note that the membership set Λ_{n} is the minimal set estimate of Θ_{0} at time n, if we choose the magnitude of the error upper bound γ properly, the membership set is nonempty. Thus, the set-membership adaptive scheme can be incorporated into the IIR-SAF-NLMM to seek the valid estimates of combined vectors (w,q) which lie in the membership set at the steady-state.
where θ is a constant, g(n)=γsgn[e(n)], γ≥0 is the set-membership error bound, and sgn[·] is the sign function.
where ε_{0} is a small regular parameter for preventing from zero division. For the special case e(n)=0 and ψ[ e(n)]=0, the weight updating is suspended.
It is noted that in (25) and (27), θα_{n} is equivalent to the step size in the IIR-SAF-NLMM, i.e., the step sizes μ_{w}=μ_{q}=θα_{n} for the IIR-SAF-SMNLMM are not constants any more. Furthermore, the IIR-SAF-SMNLMM algorithm can be viewed as the variable step size IIR-SAF-NLMM algorithm. When the upper bound γ is set to be 0, then resulting into α_{n}=1, the SAF-SMNLMM algorithm degenerates into the SAF-NLMM.
where A_{e}(n)=[|e(n)|,|e(n−1)|,⋯,|e(n−N_{w}+1)|], and λ_{1} is the forgetting factor approaching but smaller than one.
4 Computational complexity
Comparison of the computational complexities
Algorithm | Multiplications | Additions | Divisions | Median operation |
---|---|---|---|---|
IIR-SAF-LMS [9] | 2M+4N+4K_{p} | 2M+4N+4K_{q} | 0 | 0 |
IIR-SAF-NLMS [18] | 2M+4N+4K_{p}+4 | 2M+4N+4K_{q}+4 | 2 | 0 |
IIR-SAF-NLMM | 2M+4N+4K_{p}+7 | 2M+4N+4K_{q}+5 | 2 | O(N_{w} log2N_{w}) |
IIR-SAF-SMNLMM | 2M+4N+4K_{p}+12 | 2M+4N+4K_{q}+7 | 2 | O(N_{w} log2N_{w}) |
5 Convergence properties
Assumption 1
The ambient noise η(n)=η_{G}(n)+η_{I}(n), where η_{G}(n) is white Gaussian background noise with zero-mean and variance \(\sigma _{G}^{2}\) and η_{I}(n) is the impulsive noise, modeled by an independent and identically distributed (i.i.d) random variable. The sequence η(n) whose variance is \(\sigma _{\eta }^{2}\) is independent of x(n) and s(n).
Assumption 2
For sufficient long IIR weight error vector, the output error e(n) is independent of \({\varphi ^{\prime }_{i} (u_{n})}\), ∥g_{n}∥^{2} and ∥Cu_{n}∥^{2} and the parameter α_{n} involved with e(n) in (24) is also independent of \({\varphi ^{\prime }_{i} (u_{n})}\), ∥g_{n}∥^{2} and ∥Cu_{n}∥^{2}.
where c_{3} is the third row of the matrix C.
where \(\sigma _{\psi (\eta)}^{2}\) denotes the variance of ψ[η(n)].
where \( {\text {cov}} \left (\Delta \mathbf {w}_{n} \right) = E\left (\Delta \mathbf {w}_{n} \Delta \mathbf {w}_{n}^{T} \right), A_{n}\! =\! \left [\left (\mathbf {c}_{3} \mathbf {q}_{i,n} \right)/\Delta x^{2} \right ] \)
\( E \left [||\mathbf {u}_{n} ||^{- 2} \varphi ^{\prime }_{i} (u_{n})\right ], B_{n}\!\! =\!\! \left [\!\left (\mathbf {c}_{3} \mathbf {q}_{i,n} \right)^{2} /\Delta x^{4} \right ] E\left [\varphi ^{\prime }_{i} (u_{n})^{2} ||\mathbf {g}_{n}||^{2} \right. \)
||u_{n}||^{−4}], and \( C_{n} = \sigma _{\psi (\eta)}^{2} E\left [\varphi ^{\prime }_{i} (u_{n})^{2} ||\mathbf {g}_{n} ||^{2} ||\mathbf {u}_{n} ||^{- 4} \right ]. \)
6 Results and discussion
where a(n) is the white Gaussian noise signal with zero-mean and unitary variance, and the parameter 0<ω<0.95 represents the level of correlation between the adjacent samples. The lengths of the MA and AR parts in the IIR adaptive filter are set to M=2 and N=3, respectively. The initial tap-weight vector for the IIR adaptive filter is w_{−1}=[1,0,...,0] with length M+N=5, while the control point vector is initially set to a straight line with a unitary slope. Only the CR-spline basis is applied in the simulations; however, similar results can also be obtained by using the B-spline basis.
The ambient noise η(n)=η_{G}(n)+η_{I}(n), where η_{G}(n) is the white Gaussian background noise and η_{I}(n) is the impulsive noise. The background noise η_{G}(n) is the zero-mean independent white Gaussian sequence with variance \(\sigma _{G}^{2}\), with 40 dB signal-to-noise ratio (SNR) which is added to the input of the unknown system. The SNR is defined as \(\text {SNR} = 10\log _{10} \left (\sigma _{x}^{2} /\sigma _{G}^{2} \right)\), where \(\sigma _{x}^{2}\) is the variance of the system input x(n). The impulsive interference η_{I}(n) is modeled by the contaminated Gaussian (CG) process or the symmetric α−S distribution. The CG impulse can be represented by η_{I}(n)=z(n)b(n) with a signal-to-interference ratio (SIR) of − 10 dB or − 20 dB, where z(n) is a white Gaussian process with zero-mean and b(n) is a Bernoulli sequence with the probability mass function with P(b)=1−P for b=0 and P(b)=P for b=1, where P is the probability of the occurrence of the impulsive interference. The SIR is defined as \(\text {SIR} = 10\log _{10} \left (\sigma _{d}^{2} /\sigma _{z}^{2} \right)\), where \(\sigma _{z}^{2}\) and \(\sigma _{d}^{2}\) are the variances of z(n) and the desired signal \(\tilde d(n),\) respectively. The symmetric α−S distribution is characterized by the fractional order parameter p and characteristic exponent α, for which the fractional-order signal-to-noise ratio (FSNR) can be defined as \(\text {FSNR} = 10\log _{10} [E(|\tilde d(n)|^{p})/E(|\eta _{I} (n)|^{p})]\) and 0<p<α. The step sizes are set to μ_{w}=μ_{q}=0.01 for the IIR-SAF-LMS, IIR-SAF-NLMS, and proposed IIR-SAF-NLMM. For the proposed SAF-SMNLMM, the constant θ is set to 0.06 except in Fig. 4. Other parameters are selected as follows: \(\gamma = \sqrt {\tau \sigma _{G}^{2}} /(\kappa + 1)\), τ=5, κ=0.6 except in Fig. 5, λ_{0}=λ_{1}=0.99, ε_{0}=ε=0.001, ω=0.7, α=0.8, and p=0.7.
Number of update ratio of the IIR-SAF-SMNLMS algorithm for different θ in the absence of impulsive noise
θ | Update ratio % |
---|---|
0.01 | 83.14 |
0.025 | 63.32 |
0.075 | 44.7 |
0.1 | 36.2 |
Number of update ratio of the IIR-SAF-SMNLMS algorithm for different κ in the absence of impulsive noise
κ | Update ratio % |
---|---|
0 | 12.21 |
0.25 | 34.59 |
0.5 | 44.49 |
0.8 | 61.08 |
Number of update ratio % for the corresponding algorithms in the CG impulsive noise
IIR-SAF-LMS | IIR-SAF-NLMS | IIR-SAF-NLMM | IIR-SAF-SMNLMM | |
---|---|---|---|---|
SIR = − 10 dB P = 0.01 | 100 | 100 | 98.89 | 49.6 |
SIR = − 10 dB P = 0.001 | 100 | 100 | 98.70 | 49.57 |
SIR = − 20 dB P = 0.001 | 100 | 100 | 98.79 | 48.49 |
7 Conclusions
In order to suppress the effect of the impulsive noise and decrease the computational burden, this paper combines the set-membership framework and least-M estimate scheme and proposes two variants based on the IIR spline adaptive filter. The proposed SAF-IIR-NLMM algorithm is derived by using a robust M-estimator as the cost function and the SAF-IIR-SMNLMM is characterized by the set-membership error bound leading into an evident decrease of the number of the update ratio. Moreover, the computational burdens and the convergence properties of the proposed SAF-IIR-SMNLMM algorithm are also given. Compared to the cited spline adaptive filtering algorithms, the proposed algorithms offer more robustness against impulsive noise, better tracking ability, and lower computational complexity.
8 Methods/Experimental
This paper studies the SAF-IIR-NLMM and SAF-IIR-SMNLMM algorithms aiming at suppressing the effect of the impulsive noise and decreasing the computational burden compared with the conventional nonlinear adaptive spline adaptive algorithms. The derivation of the algorithms are based on the modified Huber M-estimate function and set-membership framework. Besides, the convergence properties of the SAF-IIR-SMNLMM algorithm are analyzed by using the energy conversion relation. The numerical experiments are carried out by applying the white Gaussian noise signal and colored noise signal in the CG impulsive noise or symmetric α−S impulsive noise environment. The results demonstrated that the two proposed variants of the SAF are robust to the impulsive noise, and the SAF-IIR-SMNLMM algorithm obtains low updating ratio.
Notes
Acknowledgements
The authors would like to thank National Natural Science Foundation of China for financially support.
Funding
This work was financially supported by the National Natural Science Foundation of China under Grant 61501119 and by the Fund for the Dongguan Municipal Science and Technology Bureau under Grant 2016508140
Availability of data and materials
Please contact author for data requests.
Ethic approval and consent to participate
Not applicable.
Authors’ contributions
All authors contributed equally to this work. All authors read and approved the final manuscript.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.S. Haykin, Adaptive filter theory, 4th edn (Prentice-Hall, Englewood Cliffs, 2002).zbMATHGoogle Scholar
- 2.A. H. Sayed, Adaptive filters (Wiley, NJ, 2008).CrossRefGoogle Scholar
- 3.V. J. Mathews, Adaptive polynomial filters. IEEE Sig. Process. Mag. 8:, 10–26 (1991).CrossRefGoogle Scholar
- 4.S. Haykin, Neural networks and learning machines, 2th edn (Prentice-Hall, Englewood Cliffs, 2008).Google Scholar
- 5.E. W. Bai, F. Giri, Introduction to block-oriented nonlinear systems (Springer, London, 2010).CrossRefGoogle Scholar
- 6.M. Scarpiniti, D. Comminiello, R. Parisi, A. Uncini, Nonlinear spline adaptive filtering. Sig. Process.93:, 772–783 (2013).CrossRefGoogle Scholar
- 7.M. Scarpiniti, D. Comminiello, R. Parisi, A. Uncini, Hammerstein uniform cubic spline adaptive filtering: learning and convergence properties. Sig. Process.100:, 112–123 (2014).CrossRefGoogle Scholar
- 8.M. Scarpiniti, D. Comminiello, R. Parisi, A. Uncini, Novel cascade spline architectures for the identification of nonlinear systems. IEEE Trans. Circ. Syst.-I: Regular Papares.62:, 1825–1835 (2015).MathSciNetzbMATHGoogle Scholar
- 9.M. Scarpiniti, D. Comminiello, R. Parisi, A. Uncini, Nonlinear system identification using IIR spline adaptive filters. Sig. Process.108:, 30–35 (2015).CrossRefGoogle Scholar
- 10.F. Lindsten, T. B. Schon, M. I. Jordanb, Bayesian semiparametric Wiener system identification. Automatica. 49:, 2053–2063 (2013).MathSciNetCrossRefGoogle Scholar
- 11.M. Rasouli, D. Westwick, W. Rosehart, Quasiconvexity analysis of the Hammerstein model. Automatica. 50:, 277–281 (2014).MathSciNetCrossRefGoogle Scholar
- 12.A. E. Nordsjo, L. H. Zetterberg, Identification of certain time-varying Wiener and Hammerstein systems. IEEE Trans. Signal Process.49:, 577–592 (2001).CrossRefGoogle Scholar
- 13.M. A. Chitre, J. R. Potter, S. H. Ong, Optimal and near-optimal signal detection in snapping shrimp dominated ambient noise. IEEE Ocean. J. Eng. 31:, 497–503 (2006).CrossRefGoogle Scholar
- 14.K. J. Sangston, K. R. Gerlach, Non-Gaussian noise models and coherent detection of radar targets. IEEE Trans. Aerosp. Electron. Syst.30:, 330–340 (1992).CrossRefGoogle Scholar
- 15.A. Mahmood, M. A. Chitre, M. A. Armand, Detecting OFDM signals in alpha-stable noise. IEEE Trans. Commun.62:, 3571–3583 (2014).CrossRefGoogle Scholar
- 16.S. C. Chan, Y. X. Zou, A recursive least M-estimate algorithm for robust adaptive filtering in impulsive noise: fast algorithm and convergence performance analysis. IEEE Trans. Sig. Procecss.52:, 975–991 (2004).MathSciNetCrossRefGoogle Scholar
- 17.Y. Zou, S. C. Chan, T. S. Ng, Lesat mean M-estiamte algorithms for robust adaptive filtering in impulse noise. IEEE Trans. Circ. Syst.-II: Analog. Digit. Sig. Process.47:, 1564–1569 (2000).CrossRefGoogle Scholar
- 18.S. Guan, Z. Li, Normalised spline adaptive filtering algorithm for nonlinear system identification. Neural Process. Lett.5:, 1–13 (2017).Google Scholar
- 19.S. Gollamudi, S. Nagaraj, S. Kapoor, Y. F. Huang, Set-membership filtering and a set-membership normalized LMS algorithm with an adaptive step size. IEEE Sig. Process. Lett. 5:, 111–114 (1998).CrossRefGoogle Scholar
- 20.S Zhang, J Zhang, H Han, Robust shrinkage normalized sign algorithm in an impulsive noise environment. IEEE Trans. Circuits Syst.-II: Express Briefs. 64:, 91–95 (2017).CrossRefGoogle Scholar
- 21.S. Guarnieri, F. Piazza, A. Uncini, Multilayer feedforward networks with adaptive spline activation function. IEEE Trans. Neural Netw.10:, 672–683 (1999).CrossRefGoogle Scholar
- 22.M. Scarpiniti, D. Comminiello, G. Scarano, R. Parisi, A. Uncini, Steady-state performance of spline adaptive filters. IEEE Trans. Sig. Process.64:, 816–828 (2016).MathSciNetCrossRefGoogle Scholar
- 23.Z. Zheng, H. Zhao, Affine projection M-estiamte subband adaptive filters for robust adaptive filtering in impulsive noise. Sig. Process.120:, 64–70 (2016).CrossRefGoogle Scholar
Copyright information
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.