Exact Quadratic Error of the Local Linear Regression Operator Estimator for Functional Covariates

Abstract

In this paper, it is studied the asymptotic behavior of the nonparametric local linear estimation of the regression operator when the covariates are curves. Under some general conditions we give the exact expression involved in the leading terms of the quadratic error of this estimator. The obtained results affirm the superiority of the local linear modeling over the kernel method, in functional statistics framework.

Appendix: Proofs

In what follows, when no confusion is possible, we will denote by C and C′ some strictly positive generic constants. Moreover, we set for all i, j = 1, …, n and for a fixed $(x,y) \in \mathcal{F}\times \mathrm{I\!R}$:

$$\displaystyle{K_{i} = K(h^{-1}\delta (x,X_{ i})),\;\beta _{i} =\beta (X_{i},x)\mbox{ and }W_{ij}(x) = W_{ij}.}$$

Proof of Lemma 1

We have:

$$\displaystyle\begin{array}{rcl} \mathbb{E}\left [\hat{g}(x)\right ]& =& \mathbb{E}\left [ \frac{1} {n(n - 1)\mathbb{E}[W_{12}]}\sum _{j\not =i,1}^{n}W_{ ij}Y _{j}\right ] \\ & =& \frac{\mathbb{E}[W_{12}Y _{2}]} {\mathbb{E}[W_{12}]} = \frac{1} {\mathbb{E}[W_{12}]}\mathbb{E}\left [W_{12}\mathbb{E}[Y _{2}\vert X_{2}]\right ].{}\end{array}$$

(3)

Then, it follows from (3) and the definition of the operator m that:

$$\displaystyle{\mathbb{E}\left [\hat{g}(x)\right ] = \frac{1} {\mathbb{E}[W_{12}]}\mathbb{E}\left [W_{12}m(X_{2})\right ].}$$

Now, by the same arguments as those used in [13], for the regression operator estimation, we show that:

$$\displaystyle\begin{array}{rcl} \mathbb{E}\left [W_{12}m(X_{2})\right ]& =& m(x)\mathbb{E}[W_{12}] + \mathbb{E}\left [W_{12}\left (m(X_{2}) - m(x)\right )\right ] {}\\ & =& m(x)\mathbb{E}[W_{12}] + \mathbb{E}\left [W_{12}\mathbb{E}\left [m(X_{2}) - m(x)\vert \beta (x,X_{2})\right ]\right ] {}\\ & =& m(x)\mathbb{E}[W_{12}] + \mathbb{E}\left [W_{12}\varPsi \left (\beta (x,X_{2})\right )\right ] {}\\ \end{array}$$

and since $\mathbb{E}\left [\beta (x,X_{2})W_{12}\right ] = 0$ and Ψ(0) = 0, we obtain:

$$\displaystyle{\begin{array}{cc} \mathbb{E}\left [W_{12}\varPsi \left (\beta (x,X_{2})\right )\right ]& = \frac{1} {2}\varPsi ^{{\prime\prime}}(0)\mathbb{E}\left [\beta ^{2}(x,X_{ 2})W_{12}\right ] + o\left (\mathbb{E}\left [\beta ^{2}(x,X_{ 2})W_{12}\right ]\right ). \end{array} }$$

Then:

$$\displaystyle{ \mathbb{E}\left [\hat{g}(x)\right ] = m(x) +\varPsi _{ 0}^{{\prime\prime}}(0)\frac{\mathbb{E}\left [\beta ^{2}(x,X_{ 2})W_{12}\right ]} {2\mathbb{E}[W_{12}]} + o\left (\frac{\mathbb{E}\left [\beta ^{2}(x,X_{2})W_{12}\right ]} {\mathbb{E}[W_{12}]} \right ). }$$

Moreover, it is clear that:

$$\displaystyle\begin{array}{rcl} \mathbb{E}\left [\beta (x,X_{2})^{2}W_{ 12}\right ]& =& \left (\mathbb{E}\left [K_{1}\beta _{1}^{2}\right ]\right )^{2} - \mathbb{E}[K_{ 1}\beta _{1}]\mathbb{E}[K_{1}\beta _{1}^{3}] {}\\ \mathbb{E}\left [W_{12}\right ]& =& \mathbb{E}[K_{1}\beta _{1}^{2}]EK_{ 1} - (\mathbb{E}[K_{1}\beta _{1}])^{2} {}\\ \end{array}$$

and, under the Assumption (H4), we obtain that:

$$\displaystyle{\mbox{ for all }a> 0,\; \mathbb{E}[K_{1}^{a}\beta _{ 1}] \leq C\int _{B(x,h)}\beta (u,x)dP_{X}(u).}$$

So, by using the last part of the Assumption (H3), we get:

$$\displaystyle{h\mathbb{E}[K_{1}^{a}\beta _{ 1}] = o\left (\int _{B(x,h)}\beta ^{2}(u,x)dP_{ X}(u)\right ) = o(h^{2}\phi _{ x}(h))}$$

which allows to write:

$$\displaystyle{ \mathbb{E}[K_{1}^{a}\beta _{ 1}] = o(h\phi _{x}(h)). }$$

(4)

Moreover, for all b > 1, we can write:

$$\displaystyle{\mathbb{E}[K_{1}^{a}\beta _{ 1}^{b}] = \mathbb{E}[K_{ 1}^{a}\delta ^{b}(x,X_{ 1})] + \mathbb{E}\left [K_{1}(\beta ^{b}(X_{ 1},x) -\delta ^{b}(x,X_{ 1}))\right ].}$$

Then, the second part of the Assumption (H3) implies that:

$$\displaystyle\begin{array}{rcl} & & \mathbb{E}\left [K_{1}^{a}(\beta ^{b}(X_{ 1},x) -\delta ^{b}(x,X_{ 1}))\right ] {}\\ & & \ \ = \mathbb{E}\left [K_{1}^{a}I_{ B(x,h)}(\beta (X_{1},x) -\delta (x,X_{1}))\sum _{l=0}^{b-1}(\beta (X_{ 1},x))^{b-1-l}(\delta (x,X_{ 1}))^{l}\right ] {}\\ & & \ \ \leq \sup _{u\in B(x,h)}\vert \beta (u,x) -\delta (x,u)\vert \sum _{l=0}^{b-1}\mathbb{E}\left [K_{ 1}^{a}I_{ B(x,h)}\vert \beta (X_{1},x)\vert ^{b-1-l}\vert \delta (x,X_{ 1})\vert ^{l})\right ], {}\\ \end{array}$$

whereas the first part of the Assumption (H3) gives:

$$\displaystyle{I_{B(x,h)}\vert \beta (X_{1},x)\vert \leq I_{B(x,h)}\vert \delta (x,X_{1})\vert.}$$

Thus, it follows:

$$\displaystyle\begin{array}{rcl} \mathbb{E}\left [K_{1}^{a}(\beta ^{b}(X_{ 1},x) -\delta ^{b}(x,X_{ 1}))\right ]& \leq & b\sup _{u\in B(x,h)}\vert \beta (u,x) -\delta (x,u)\vert \vert \mathbb{E}[K_{1}^{a}\vert \delta \vert ^{b-1}(x,X_{ 1})] {}\\ & \leq & b\sup _{u\in B(x,h)}\vert \beta (u,x) -\delta (x,u)\vert h^{b-1}\mathbb{E}[K_{ 1}^{a}] {}\\ & \leq & b\sup _{u\in B(x,h)}\vert \beta (u,x) -\delta (x,u)\vert h^{b-1}\phi _{ x}(h) {}\\ \end{array}$$

which allows to write:

$$\displaystyle{\mathbb{E}[K_{1}^{a}\beta _{ 1}^{b}] = \mathbb{E}[K_{ 1}^{a}\delta ^{b}(x,X_{ 1})] + o(h^{b}\phi _{ x}(h)).}$$

Concerning the term $\mathbb{E}[K_{1}^{a}\delta ^{b}]$, we write:

$$\displaystyle\begin{array}{rcl} h^{-b}\mathbb{E}[K_{ 1}^{a}\delta ^{b}]& =& \int v^{b}K^{a}(v)dP_{ X}^{h^{-1}\delta (x,X_{ 1})}(v) {}\\ & =& \int _{-1}^{1}\left [K^{a}(1) -\int _{ v}^{1}(u^{b}K^{a}(u))^{{\prime}}du\right ]dP_{ X}^{h^{-1}\delta (x,X_{ 1})}(v) {}\\ & =& K^{a}(1)\phi _{ x}(h) -\int _{-1}^{1}(u^{b}K^{a}(u))^{{\prime}}\phi _{ x}(uh,h)du {}\\ & =& \phi _{x}(h)\left (K^{a}(1) -\int _{ -1}^{1}(u^{b}K^{a}(u))^{{\prime}}\frac{\phi _{x}(uh,h)} {\phi _{x}(h)} du\right ). {}\\ \end{array}$$

Finally, under the Assumption (H1), we get:

$$\displaystyle{ \mathbb{E}[K_{1}^{a}\beta _{ 1}^{b}] = h^{b}\phi _{ x}(h)\left (K^{a}(1) -\int _{ -1}^{1}(u^{b}K^{a}(u))^{{\prime}}\chi _{ x}(u)du\right ) + o(h^{b}\phi _{ x}(h)). }$$

(5)

It follows that:

$$\displaystyle\begin{array}{rcl} \frac{\mathbb{E}\left [\beta ^{2}(x,X_{2})W_{12}\right ]} {\mathbb{E}[W_{12}]} & =& h^{2}\left [\frac{K(1) -\int _{-1}^{1}(u^{2}K(u))^{{\prime}}\chi _{ x}(u)du} {K(1) -\int _{-1}^{1}K^{{\prime}}(u)\chi _{ x}(u)du} \right ] + o(h^{2}). {}\\ \end{array}$$

Consequently:

$$\displaystyle\begin{array}{rcl} \mathbb{E}\left [\hat{g}(x)\right ]& =& m(x) + \frac{h^{2}} {2} \varPsi _{0}^{{\prime\prime}}(0)\left [\frac{K(1) -\int _{-1}^{1}(u^{2}K(u))^{{\prime}}\chi _{ x}(u)du} {K(1) -\int _{-1}^{1}K^{{\prime}}(u)\chi _{ x}(u)du} \right ] + o(h^{2}). {}\\ \end{array}$$

 ■ 

Proof of Lemma 2

For this Lemma, we use the same ideas of Sarda and Vieu [16] to show that

$$\displaystyle\begin{array}{rcl} \mathrm{Var}\left [\hat{m}(x)\right ]& =& \mathrm{Var}\left [\hat{g}(x)\right ] - 2(\mathbb{E}\hat{g}(x))\mathrm{Cov}(\hat{g}(x),\hat{f}(x)) {}\\ & & +(\mathbb{E}\hat{g}(x))^{2}\mathrm{Var}(\,\hat{f}(x)) + o\left ( \frac{1} {n\phi _{x}(h)}\right ). {}\\ \end{array}$$

It is clear that:

$$\displaystyle\begin{array}{rcl} \mbox{ Var}\left (\hat{g}(x)\right )& =& \frac{1} {(n(n - 1)\mathbb{E}[W_{12}])^{2}}\mbox{ Var}\left (\sum _{i\not =j\,=1}^{n}W_{ ij}Y _{j}\right ) {}\\ & =& \frac{1} {\left (n(n - 1)(EW_{12})\right )^{2}}\left (n(n - 1)\mathbb{E}[W_{12}^{2}Y _{ 2}^{2}] + n(n - 1)\mathbb{E}[W_{ 12}W_{21}Y _{2}Y _{1}]\right. {}\\ & & \left.+n(n - 1)(n - 2)\mathbb{E}[W_{12}W_{13}Y _{2}Y _{3}] + n(n - 1)(n - 2)\mathbb{E}[W_{12}W_{23}Y _{2}Y _{3}]\right. {}\\ & & \left.+n(n - 1)(n - 2)\mathbb{E}[W_{12}W_{31}Y _{2}Y _{1}] + n(n - 1)(n - 2)\mathbb{E}[W_{12}W_{32}Y _{2}^{2}]\right. {}\\ & & \left.-n(n - 1)(4n - 6)(\mathbb{E}[W_{12}Y _{2}])^{2}\right ). {}\\ \end{array}$$

Observe that the terms of the first line are negligible compared to other terms which are multiplied by $n(n - 1)(n - 2)$. Furthermore,

$$\displaystyle\begin{array}{rcl} \mathbb{E}[W_{12}^{2}Y _{ 2}^{2}]& =& O(h^{4}\phi _{ x}^{2}(h)), {}\\ \mathbb{E}[W_{12}W_{21}Y _{1}Y _{2}]& =& O(h^{4}\phi _{ x}^{2}(h)), {}\\ \mathbb{E}[W_{12}W_{13}Y _{2}Y _{3}]& =& (m(x))^{2}\mathbb{E}[\beta _{ 1}^{4}K_{ 1}^{2}](\mathbb{E}[K_{ 1}])^{2} + o(h^{4}\phi _{ x}^{3}(h)), {}\\ \mathbb{E}[W_{12}W_{23}Y _{2}Y _{3}]& =& (m(x))^{2}\mathbb{E}[\beta _{ 1}^{2}K_{ 1}](\mathbb{E}[\beta _{1}^{2}K_{ 1}^{2}]\mathbb{E}[K_{ 1}]) + o(h^{4}\phi _{ x}^{3}(h)), {}\\ \mathbb{E}[W_{12}W_{31}Y _{2}Y _{1}]& =& (m(x))^{2}\mathbb{E}[\beta _{ 1}^{2}K_{ 1}](\mathbb{E}[\beta _{1}^{2}K_{ 1}^{2}]\mathbb{E}[K_{ 1}]) + o(h^{4}\phi _{ x}^{3}(h)), {}\\ \mathbb{E}[W_{12}W_{32}Y _{2}^{2}]& =& (m_{ 2}(x))(\mathbb{E}[\beta _{1}^{2}K_{ 1}])^{2}(\mathbb{E}[K_{ 1}^{2}]) + o(h^{4}\phi _{ x}^{3}(h)), {}\\ \mathbb{E}[W_{12}Y _{2}]& =& O(h^{2}\phi _{ x}^{2}(h)). {}\\ \end{array}$$

Therefore, the leading term in the expression of $\mbox{ Var}\left (\hat{g}(x)\right )$ is:

$$\displaystyle\begin{array}{rcl} & & \frac{n(n - 1)(n - 2)} {(n(n - 1)\mathbb{E}[W_{12}])^{2}}\left ((m(x))^{2}\left (\mathbb{E}[\beta _{ 1}^{4}K_{ 1}^{2}](\mathbb{E}[K_{ 1}])^{2} + 2BBe[\beta _{ 1}^{2}K_{ 1}](\mathbb{E}[\beta _{1}^{2}K_{ 1}^{2}]\mathbb{E}[K_{ 1}])\right )\right. {}\\ & & \left.\qquad + (m_{2}(x))(\mathbb{E}[\beta _{1}^{2}K_{ 1}])^{2}(\mathbb{E}[K_{ 1}^{2}]) + o(h^{4}\phi _{ x}^{3}(h))\right ). {}\\ \end{array}$$

Concerning the covariance term, we have by the same fashion:

$$\displaystyle\begin{array}{rcl} \text{Cov}(\hat{g}(x),\hat{f}(x))& =& \frac{1} {(n(n - 1)\mathbb{E}[W_{12}])^{2}}\text{Cov}\left (\sum _{\stackrel{i,j=1}{i\not =j}}^{n}W_{ ij}Y _{j},\sum _{\stackrel{i^{{\prime}},j^{{\prime}}=1}{i^{{\prime}}\not =j^{{\prime}}}}^{n}W_{ i^{{\prime}}j^{{\prime}}}\right ) {}\\ & =& \frac{1} {\left (n(n - 1)EW_{12})\right )^{2}}\left [n(n - 1)\mathbb{E}[W_{12}^{2}Y _{ 2}] + n(n - 1)\mathbb{E}[W_{12}W_{21}Y _{2}]\right. {}\\ & & \left.+n(n - 1)(n - 2)\mathbb{E}[W_{12}W_{13}Y _{2}] + n(n - 1)(n - 2)\mathbb{E}[W_{12}W_{23}Y _{2}]\right. {}\\ & & \left.+n(n - 1)(n - 2)\mathbb{E}[W_{12}W_{31}Y _{2}] + n(n - 1)(n - 2)\mathbb{E}[W_{12}W_{32}Y _{2}]\right. {}\\ & & \left.-n(n - 1)(4n - 6)(\mathbb{E}[W_{12}Y _{2}]\mathbb{E}[W_{12}])\right ] {}\\ \end{array}$$

with

$$\displaystyle\begin{array}{rcl} \mathbb{E}[W_{12}^{2}Y _{ 2}]& =& O(h^{4}\phi _{ x}^{2}(h)), {}\\ \mathbb{E}[W_{12}W_{21}Y _{2}]& =& O(h^{4}\phi _{ x}^{2}(h)), {}\\ \mathbb{E}[W_{12}W_{13}Y _{2}]& =& (m(x))\mathbb{E}[\beta _{1}^{4}K_{ 1}^{2}](\mathbb{E}[K_{ 1}])^{2} + o(h^{4}\phi _{ x}^{3}(h)), {}\\ \mathbb{E}[W_{12}W_{23}Y _{2}]& =& (m(x))\mathbb{E}[\beta _{1}^{2}K_{ 1}](\mathbb{E}[\beta _{1}^{2}K_{ 1}^{2}]\mathbb{E}[K_{ 1}]) + o(h^{4}\phi _{ x}^{3}(h)), {}\\ \mathbb{E}[W_{12}W_{31}Y _{2}]& =& (m(x))\mathbb{E}[\beta _{1}^{2}K_{ 1}](\mathbb{E}[\beta _{1}^{2}K_{ 1}^{2}]\mathbb{E}[K_{ 1}]) + o(h^{4}\phi _{ x}^{3}(h)), {}\\ \mathbb{E}[W_{12}W_{32}Y _{2}]& =& (m(x))(\mathbb{E}[\beta _{1}^{2}K_{ 1}])^{2}(\mathbb{E}[K_{ 1}^{2}]) + o(h^{4}\phi _{ x}^{3}(h)), {}\\ \mathbb{E}[W_{12}Y _{1}]& =& O(h^{2}\phi _{ x}^{2}(h)). {}\\ \end{array}$$

Therefore, the leading term in the expression of $\text{Cov}(\hat{g}(x),\hat{f}(x))$ is:

$$\displaystyle\begin{array}{rcl} & & \frac{n(n - 1)(n - 2)} {(n(n - 1)\mathbb{E}[W_{12}])^{2}}\left (m(x)\left (\mathbb{E}[\beta _{1}^{4}K_{ 1}^{2}](\mathbb{E}[K_{ 1}])^{2} + 2\mathbb{E}[\beta _{ 1}^{2}K_{ 1}](\mathbb{E}[\beta _{1}^{2}K_{ 1}^{2}]\mathbb{E}[K_{ 1}])\right.\right. {}\\ & & \left.\left.\qquad + (\mathbb{E}[\beta _{1}^{2}K_{ 1}])^{2}(\mathbb{E}[K_{ 1}^{2}])\right ) + o(h^{4}\phi _{ x}^{3}(h))\right ). {}\\ \end{array}$$

Finally, for $\text{Var}\left (\hat{f}(x)\right )$

$$\displaystyle\begin{array}{rcl} \text{Var}\left (\hat{f}(x)\right )& =& \frac{1} {\left (n(n - 1)(\mathbb{E}W_{12})\right )^{2}}\Big[n(n - 1)\mathbb{E}[[W_{12}^{2}] + n(n - 1)\mathbb{E}[[W_{ 12}W_{21}] {}\\ & & \left.+n(n - 1)(n - 2)\mathbb{E}[[W_{12}W_{13}] + n(n - 1)(n - 2)\mathbb{E}[[W_{12}W_{23}]\right. {}\\ & & \left.+n(n - 1)(n - 2)\mathbb{E}[[W_{12}W_{31}] + n(n - 1)(n - 2)\mathbb{E}[[W_{12}W_{32}]\right. {}\\ & & -n(n - 1)(4n - 6)(\mathbb{E}[[W_{12}])^{2}\Big] {}\\ \end{array}$$

and similarly to the previous cases:

$$\displaystyle\begin{array}{rcl} \mathbb{E}[W_{12}^{2}]& =& O(h^{4}\phi _{ x}^{2}(h)), {}\\ \mathbb{E}[[W_{12}W_{21}]& =& O(h^{4}\phi _{ x}^{2}(h)), {}\\ \mathbb{E}[[W_{12}W_{13}]& =& \mathbb{E}[[\beta _{1}^{4}K_{ 1}^{2}](\mathbb{E}[[K_{ 1}])^{2} + o(h^{4}\phi _{ x}^{3}(h)), {}\\ \mathbb{E}[[W_{12}W_{23}]& =& \mathbb{E}[[\beta _{1}^{2}K_{ 1}](\mathbb{E}[[\beta _{1}^{2}K_{ 1}^{2}]\mathbb{E}[[K_{ 1}]) + o(h^{4}\phi _{ x}^{3}(h)), {}\\ \mathbb{E}[[W_{12}W_{31}]& =& \mathbb{E}[[\beta _{1}^{2}K_{ 1}](\mathbb{E}[[\beta _{1}^{2}K_{ 1}^{2}]\mathbb{E}[[K_{ 1}]) + o(h^{4}\phi _{ x}^{3}(h)), {}\\ \mathbb{E}[[W_{12}W_{32}]& =& (\mathbb{E}[[\beta _{1}^{2}K_{ 1}])^{2}(\mathbb{E}[[K_{ 1}^{2}]) + o(h^{4}\phi _{ x}^{3}(h)), {}\\ \mathbb{E}[[W_{12}]& =& O(h^{2}\phi _{ x}^{2}(h)). {}\\ \end{array}$$

Therefore,

$$\displaystyle\begin{array}{rcl} \text{Var}\left (\hat{m}(x)\right ) = \frac{(m_{2}(x) - m^{2}(x))} {n\phi _{x}(h)} \left [\frac{\left (K^{2}(1) -\int _{-1}^{1}(K^{2}(u))^{{\prime}}\chi (u)du\right )} {\left (K(1) -\int _{-1}^{1}(K(u))^{{\prime}}\chi (u)du\right )^{2}} \right ] + o\left ( \frac{1} {n\phi _{x}(h)}\right ).& & {}\\ \end{array}$$

which completes the proof. ■