A Data-Driven Basis Function Approach in Nonparametric Nonlinear System Identification

Abstract

In this chapter, a data-driven orthogonal basis function approach is proposed for nonparametric FIR nonlinear system identification. The basis functions are not fixed a priori and match the structure of the unknown system automatically. This eliminates the problem of blindly choosing the basis functions without a priori structural information. Further, based on the proposed basis functions, approaches are proposed for model order determination and regressor selection along with their theoretical justifications. Both random inputs and deterministic inputs are considered.

In memory of Dr Roberto Tempo.

Appendix

Proof of Theorem 1: The first part is directly from the definition of $\phi _i$’s. Also from the definition, it is easily verified that $\mathbf{E}\phi _j[k]=0$ for $j=1,...,n$. $\mathbf{E}\phi _j[k]=0$, $j=n+1,...,M$ follows from $\mathbf{E}f_{j_1j_2}(u[k-j_1],u[k-j_2])=0$. We now show $\mathbf{E}\phi _{j_1}[k]\phi _{j_2}[k]=0$. For $0\le j_1 < j_2 \le n$, $\mathbf{E}\phi _{j_1}[k]\phi _{j_2}[k]=\mathbf{E}\phi _{j_1}[k] \mathbf{E}\phi _{j_2}[k]=0$ because of independence of $u[k-j_1]$ and $u[k-j_2]$. The proofs for other $j_1$ and $j_2$ follow from the same arguments as

$$\begin{aligned} \mathbf{E}\phi _1[k]\phi _{n+1}[k]&=\mathbf{E}\phi _1(u[k-1])\phi _{n+1}(u[k-1],u[k-2]) \\&=\mathbf{E}\{ \phi _1(u[k-1]) \mathbf{E}\{\phi _{n+1}(u[k-1],u[k-2])~|~u[k-1]\}\}=0. \end{aligned}$$

To show the third part, observe

$$\begin{aligned} y[k]=c+\sum _{j=1}^n f_j(u[k-j])+\sum _{1\le j_1 <j_2 \le n} f_{j_1j_2}(u[k-j_1],u[k-j_2])+v[k], \end{aligned}$$

$$\begin{aligned} \mathbf{E}y[k]&= c=\phi _0 \\ \mathbf{E}\{y[k]|u[k-j]=x_j\}&=c+f_j(x_j)=\phi _0+\phi _j(x_j),~j=1,...,n \\ \mathbf{E}\{y[k]~|~u[k-j_1]&=x_{j_1},u[k-j_2]=x_{j_2}\} \\&=c+f_{j_1}(x_{j_1})+f_{j_2}(x_{j_2})+f_{j_1j_2}(x_{j_1},x_{j_2}) \\&=\phi _0+\phi _{j_1}(x_{j_1}) +\phi _{j_2}(x_{j_2})+f_{j_1j_2}(x_{j_1},x_{j_2}),~1\le j_1<j_2 \le n \end{aligned}$$

Then, the conclusion follows from the definition of $\phi _j$’s.

Proof of Theorem 2: The first part is from Theorem 1 and the law of large numbers,

$$ \widehat{\phi }_0={1 \over N} \sum y[k] \rightarrow \mathbf{E}y[k]=\phi _0. $$

For the second part, from the assumptions $\delta \rightarrow 0$, $\delta N \rightarrow \infty$ as $N \rightarrow \infty$, the number of samples $u[k-j]$’s in the interval,

$$ \varphi _j(x_j,k)=|u[k-j]-x_j| \le \delta $$

converges to $2\psi (x_j)\delta N \rightarrow \infty$, where the probability density function of the input at $x_j$, $\psi (x_j)$, is assumed to be positive, or the number of elements $l_j \rightarrow 2\psi (x_j)\delta N \rightarrow \infty$. Now,

$$\begin{aligned}&|\widehat{\phi }_j(x_j) -\phi _j(x_j)|= |\sum _{k=1}^{N} w_j(x_j,k)y[k]- \phi _j(x_j) -\widehat{\phi }_0| \\&=|\sum _{k=1}^N w_j(x_j,k) (\phi _0 -\widehat{\phi }_0)+ \sum _{k=1}^{N} w_j(x_j,k) (\phi _j(u[k-j])-\phi _j(x_j)) \end{aligned}$$

$$\begin{aligned}&+\sum _{i=1,i\not =j}^n\sum _{k=1}^{N} w_j(x_j,k) \phi _i(u[k-i]) +\sum _{j=n+1}^M\sum _{k=1}^{N} w_j(x_j,k) \phi _j[k]+\sum _{k=1}^{N} w_j(x_j,k)v[k]| \\&=|\sum _{l=1}^{l_j} w_j(x_j,m_j(l)) (\phi _0 -\widehat{\phi }_0)+ \sum _{l=1}^{l_j} w_j(x_j,m_j(l)) (\phi _j(u[m_j(l)-j])-\phi _j(x_j)) \\&+ \sum _{i=1,i\not =j}^n\sum _{l=1}^{l_j} w_j(x_j,m_j(l)) \phi _i(u[m_j(l)-i]) +\sum _{j=n+1}^M\sum _{l=1}^{l_j} w_j(x_j,m_j(l)) \phi _j[m_j(l)]+ \\&\sum _{l=1}^{l_j} w_j(x_j,m_j(l))v[m_j(l)]| \le |\sum _{l=1}^{l_j} w_j(x_j,m_j(l)) (\phi _0 -\widehat{\phi }_0)| \\&+|\sum _{l=1}^{l_j} |w_j(x_j,m_j(l)) (\phi _j(u[m_j(l)-j])-\phi _j(x_j))| \\&+|\sum _{i=1,i\not =j}^n\sum _{l=1}^{l_j} w_j(x_j,m_j(l)) \phi _i(u[m_j(l)-i])| \\&+|\sum _{j=n+1}^M\sum _{l=1}^{l_j} w_j(x_j,m_j(l)) \phi _j[m_j(l)]| +|\sum _{l=1}^{l_j} w_j(x_j,m_j(l))v[m_j(l)]| \end{aligned}$$

With L being the Lipschitz constant and from the orthogonal properties of $\phi _j$, $l_j \rightarrow \infty$, $w_j(x_j,m_j(l)) \ge 0$ and $\sum _{l=1}^{l_j}w_j(x_j,m_j(l))=1$,

$$\begin{aligned}&\sum _{l=1}^{l_j} w_j(x_j,m_j(l)) (\phi _0 -\widehat{\phi }_0)=\phi _0 -\widehat{\phi }_0, \\&|\sum _{l=1}^{l_j} |w_j(x_j,m_j(l)) (\phi _j(u[m_j(l)-j])-\phi _j(x_j))|\le \delta L, \\&|\sum _{i=1,i\not =j}^n\sum _{l=1}^{l_j} w_j(x_j,m_j(l)) \phi _i(u[m_j(l)-i])|^2 \\&\rightarrow | \sum _{i=1,i\not = j}^n \mathbf{E}\{\phi _i(u[k-i])~|~u[k-j]=x_j\} |^2+O(\frac{1}{\delta N}), \\&|\sum _{j=n+1}^M\sum _{l=1}^{l_j} w_j(x_j,m_j(l)) \phi _j[m_j(l)]|^2 \\&\rightarrow |\sum _{j=n+1}^M \mathbf{E}\{\phi _j[k] ~|~u[k-j]=x_j\}|^2+O(\frac{1}{\delta N}), \\&|\sum _{l=1}^{l_j} w_j(x_j,m_j(l))v[m_j(l)]|^2 \rightarrow |\mathbf{E}v[k]|^2+O(\frac{1}{N}). \end{aligned}$$

Therefore,

$$ |\widehat{\phi }_j(x_j) -\phi _j(x_j)| \rightarrow |\phi _0-\widehat{\phi }_0|+ \delta L+ O(\frac{1}{\sqrt{\delta N}}) \rightarrow 0,~j=1,...,n $$

This completes the proof of the second part. The proofs of the third part are similar. The only difference is that the convergence rate is $O(\frac{1}{ \sqrt{\delta ^2 N}})$ as $N \rightarrow \infty$.

Proof of Theorem 3: It is easily verified that

$$ \int _{\infty }^\infty |K(x)| dx< \infty ,~ \int _{-\infty }^\infty | \int _{-\infty }^\infty K(x) e^{-j\omega x}dx| d \omega < \infty . $$

The rest part of the proof follows directly from Lemma 2 of [19].