Optimal model averaging estimator for semi-functional partially linear models


There have been many papers on frequentist model averaging over the past decade, but very little attention has been paid to how to conduct frequentist model averaging in functional data analysis. The present paper considers an optimal model averaging estimator for a semi-functional partially linear model with heteroscedasticity. Mallows-type and generalized cross-validation weight choice criteria are developed to assign model averaging weights. Under some regular assumptions, the resulting model averaging estimators are proved to be asymptotically optimal. Simulation results demonstrate the finite-sample performance of the proposed methods, and an empirical application with \(\hbox {PM}_{2.5}\) data illustrates the proposed estimates.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4


  1. Akaike H (1973) Maximum likelihood identification of Gaussian autoregressive moving average models. Biometrika 60:255–265

    MathSciNet  Article  Google Scholar 

  2. Ando T, Li KC (2014) A model-averaging approach for high-dimensional regression. J Am Stat Assoc 109:254–265

    MathSciNet  Article  Google Scholar 

  3. Ando T, Li KC (2017) A weight-relaxed model averaging approach for high-dimensional generalized linear models. Ann Stat 45:2654–2679

    MathSciNet  Article  Google Scholar 

  4. Andrews DW (1991) Asymptotic optimality of generalized CL, cross-validation, and generalized cross-validation in regression with heteroskedastic errors. J Econ 47:359–377

    Article  Google Scholar 

  5. Aneiros G, Vieu P (2006) Semi-functional partial linear regression. Stat Prob Lett 76:1102–1110

    MathSciNet  Article  Google Scholar 

  6. Aneiros G, Ferraty F, Vieu P (2011) Variable selection in semi-functional regression models. In: Recent advances in functional data analysis and related topics. Contributions to Statistics. Physica-Verlag, Heidelberg, pp 17–22

    Google Scholar 

  7. Aneiros G, Vilar JM, Cao R, San Roque AM (2013) Functional prediction for the residual demand in electricity spot markets. IEEE Trans Power Syst 28:4201–4208

    Article  Google Scholar 

  8. Aneiros G, Ling N, Vieu P (2015a) Error variance estimation in semi-functional partially linear regression models. J Nonparametr Stat 27:316–330

    MathSciNet  Article  Google Scholar 

  9. Aneiros G, Ferraty F, Vieu P (2015b) Variable selection in partial linear regression with functional covariate. Statism 49:1322–1347

    MathSciNet  Article  Google Scholar 

  10. Aneiros G, Raña P, Vieu P, Vilar J (2018) Bootstrap in semi-functional partial linear regression under dependence. Test 27:659–679

    MathSciNet  Article  Google Scholar 

  11. Boente G, Vahnovan A (2017) Robust estimators in semi-functional partial linear regression models. J Multivariate Anal 154:59–84

    MathSciNet  Article  Google Scholar 

  12. Buckland ST, Burnham KP, Augustin NH (1997) Model selection: an integral part of inference. Biometrics 53:603–618

    Article  Google Scholar 

  13. Cheng TCF, Ing CK, Yu SH (2015) Toward optimal model averaging in regression models with time series errors. J Econ 189:321–334

    MathSciNet  Article  Google Scholar 

  14. Diebold FX, Mariano RS (2002) Comparing predictive accuracy. J Bus Econ Stst 20:134–144

    MathSciNet  Google Scholar 

  15. Gao Y, Zhang X, Wang S, Zou G (2016) Model averaging based on leave-subject-out cross-validation. J Econ 192:139–151

    MathSciNet  Article  Google Scholar 

  16. Hansen BE (2007) Least squares model averaging. Econometrica 75:1175–1189

    MathSciNet  Article  Google Scholar 

  17. Hansen BE (2008) Least-squares forecast averaging. J Econ 146:342–350

    MathSciNet  Article  Google Scholar 

  18. Hansen BE, Racine JS (2012) Jackknife model averaging. J Econ 167:38–46

    MathSciNet  Article  Google Scholar 

  19. Kong D, Xue K, Yao F, Zhang HH (2016) Partially functional linear regression in high dimensions. Biometrika 103:147–159

    MathSciNet  Article  Google Scholar 

  20. Li KC (1987) Asymptotic optimality for \(C_p, C_L \), cross-validation and generalized cross-validation: Discrete index set. Ann Stat 15:958–975

    Article  Google Scholar 

  21. Mallows CL (1973) Some comments on Cp. Technometrics 15:661–675

    MATH  Google Scholar 

  22. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6:461–464

    MathSciNet  Article  Google Scholar 

  23. Speckman P (1998) Kernel smoothing in partial linear models. J R Stat Soc B Met 50:413–436

    MathSciNet  MATH  Google Scholar 

  24. Wan ATK, Zhang X, Zou G (2010) Least squares model averaging by Mallows criterion. J Econ 156:277–283

    MathSciNet  Article  Google Scholar 

  25. Whittle P (1960) Bounds for the moments of linear and quadratic forms in independent variables. Theor Probab Appl 5:302–305

    MathSciNet  Article  Google Scholar 

  26. Zhang X, Wang W (2019) Optimal model averaging estimation for partially linear models. Stat Sin 29:693–718

    MathSciNet  MATH  Google Scholar 

  27. Zhang X, Wan ATK, Zou G (2013) Model averaging by jackknife criterion in models with dependent data. J Econ 174:82–94

    MathSciNet  Article  Google Scholar 

  28. Zhang X, Zou G, Liang H (2014) Model averaging and weight choice in linear mixed-effects models. Biometrika 101:205–218

    MathSciNet  Article  Google Scholar 

  29. Zhang X, Yu D, Zou G, Liang H (2016) Optimal model averaging estimation for generalized linear models and generalized linear mixed-effects models. J Am Stat Assoc 111:1775–1790

    MathSciNet  Article  Google Scholar 

  30. Zhang X, Chiou J-M, Ma Y (2018) Functional prediction through averaging estimated functional linear regression models. Biometrika 105:945–962

    MathSciNet  MATH  Google Scholar 

  31. Zhao S, Zhang X, Gao Y (2016) Model averaging with averaging covariance matrix. Econ Lett 145:214–217

    MathSciNet  Article  Google Scholar 

  32. Zhu R, Zou G, Zhang X (2018) Optimal model averaging estimation for partial functional linear models. J Syst Sci Complex 38:777–800

    MathSciNet  MATH  Google Scholar 

Download references


Bai’s work was supported by Natural Science Foundation of China (11771268). Jiang’s work was supported by the Graduate Innovation Foundation of Shanghai University of Finance and Economics of China (Grant No. CXJJ-2019-414).

Author information



Corresponding author

Correspondence to Yang Bai.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (xlsx 76 KB)



Lemma 1

If condition 1 hold, then for \(1\le s \le S_n\), we have

$$\begin{aligned} \lambda _{\max }(\varvec{H}_{(s)}\varvec{H}_{(s)}^T) = O(\phi (h_s)^{-1}). \end{aligned}$$

Proof of Lemma 1

Denote the largest singualr value of a matrix A by \(\lambda _{max}(A)\), and by the Reisz inequality (see Speckman Speckman (1988)), we have that

$$\begin{aligned} \lambda ^2_{\max }(\varvec{K}_{(s)})\le \max \limits _i \sum \limits _{j = 1}^{n}| {K}_{(s),ij} | \max \limits _j \sum \limits _{i = 1}^{n}| {K}_{(s),ij} |. \end{aligned}$$

Note that \({K}_{(s),ij}\) is non-negative, then \(\max \nolimits _i \sum \nolimits _{j = 1}^{n}| {K}_{(s),ij} |\) is 1. From condition C.1, we have that \(\max \nolimits _{1 \le i,j \le n}|{K}_{(s),ij} |=O((n\phi (h_s))^{-1})\), which implies that \(\max \nolimits _j \sum \nolimits _{i = 1}^{n}|{K}_{(s),ij} |=O(\phi (h_s)^{-1})\). Therefore, we have that \(\lambda ^2_{\max }(\varvec{K}_{(s)})=O(\phi (h_s)^{-1})\).

Because \(\tilde{\varvec{H}}_{(s)}\) is an idempotent matrix, \(\lambda _{\max }(\tilde{\varvec{H}}_{(s)})\) = 1. Then, for \(1\le s \le S_n\), we have

$$\begin{aligned}&\lambda _{\max }(\varvec{H}_{(s)}\varvec{H}_{(s)}^T) \\&\quad \le \lambda _{\max }^2(\varvec{H}_{(s)})\\&\quad =\lambda _{\max }^2\{\tilde{\varvec{H}}_{(s)}(\varvec{I}-\varvec{K}_{(s)})+\varvec{K}_{(s)} \}\\&\quad \le [\lambda _{\max }(\tilde{\varvec{H}}_{(s)})\{1+\lambda _{\max }(\varvec{K}_{(s)})\}+\lambda _{\max }(\varvec{K}_{(s)})]^2\\&\quad =[\{1+\lambda _{\max }(\varvec{K}_{(s)})\}+\lambda _{\max }(\varvec{K}_{(s)})]^2\\&\quad = O(\phi (h_s)^{-1}). \end{aligned}$$

Proof of Theorem 1

Denote \(||\varvec{Z}||^2 = \sum \nolimits _{i=1}^{d}Z_i^2\), and \(\langle \varvec{Z}_1,\varvec{Z}_2\rangle =\sum \nolimits _{i=1}^{d}Z_{1,i}Z_{2,i}\), where \(\varvec{Z}, \varvec{Z}_1, \varvec{Z}_2\) are d-dimensional vectors composed of \(Z_{i}, Z_{1,i},Z_{2,i}\), respectively. The proof is similar to that of Theorem 1 of Wan et al. (2010). Let \(A(\varvec{\omega })=\varvec{I}-\varvec{H}(\varvec{\omega })\),then

$$\begin{aligned} C_n(\varvec{\omega })=L_n(\varvec{\omega })+||\varvec{\varepsilon }||^2+2\langle \varvec{\varepsilon },A(\varvec{\omega })\varvec{\mu }\rangle +2({{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\varvec{\Omega })-\langle \varvec{\varepsilon },\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle ). \end{aligned}$$

Theorem 1 is valid if the following hold: as n \(\rightarrow \infty \).

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\varepsilon },A(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$
$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\Omega )-\langle \varvec{\varepsilon },\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$
$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |L_n(\varvec{\omega })/R_n(\varvec{\omega })-1|\rightarrow 0. \end{aligned}$$

From the first part of condition C.2, we have

$$\begin{aligned} \lambda _{\max }{(\varvec{\Omega })}=O(1). \end{aligned}$$

Using the triangle inequality, Chebyshevs inequality, Theorem 2 of Whittle (1960), condition C.2, we obtain

$$\begin{aligned}&P\left\{ \sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\varepsilon },A(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })>\delta \right\} \\&\quad \le P\left\{ \sup \limits _{\varvec{\omega } \in H_n} \sum \limits _{s = 1}^{S_n} \omega _s |\varvec{\varepsilon }^{T}(I-\varvec{H}_{(s)})\varvec{\mu }|>\delta \xi _n\right\} \\&\quad \le P\left\{ \max \limits _{1\le s \le S_n} |\varvec{\varepsilon }^{T}(I-\varvec{H}_{(s)})\varvec{\mu }|>\delta \xi _n\right\} \\&\quad = P \left\{ \{|\langle \varvec{\varepsilon },A(\omega _1^0)\varvec{\mu }\rangle |>\delta \xi _n\} \bigcup \{|\langle \varvec{\varepsilon },A(\omega _2^0)\varvec{\mu }\rangle |>\delta \xi _n\}\right. \\&\qquad \left. \bigcup \ldots \bigcup \{|\langle \varvec{\varepsilon },A(\omega _{S_n}^0)\varvec{\mu }\rangle |>\delta \xi _n\}\right\} \\&\quad \le \sum \limits _{s = 1}^{S_n} P \left\{ |\langle \varvec{\varepsilon },A(\omega _s^0)\varvec{\mu }\rangle |>\delta \xi _n\} \right\} \\&\quad \le \sum \limits _{s = 1}^{S_n} E \left\{ \langle \varvec{\varepsilon },A(\omega _s^0)\varvec{\mu }\rangle ^{2N} / (\delta \xi _n)^{2N} \right\} \\&\quad \le C (\delta \xi _n)^{-2N}\sum \limits _{s = 1}^{S_n} || \varvec{\Omega } ^{1/2} A(\varvec{\omega }_s^0)\varvec{\mu }||^{2N} \\&\quad \le C (\delta \xi _n)^{-2N}\lambda _{\max }^{N}(\varvec{\Omega })\sum \limits _{s = 1}^{S_n} || A(\varvec{\omega }_s^0)\varvec{\mu }||^{2N} \\&\quad \le C (\delta \xi _n)^{-2N}\lambda _{\max }^{N}(\varvec{\Omega })\sum \limits _{s = 1}^{S_n} R_n(\omega _s^0)^N, \end{aligned}$$

Then, by combining (A.4) and second part of condition C.2, we have (A.1).

Similarly, we have

$$\begin{aligned}&P\left\{ \sup \limits _{\varvec{\omega } \in H_n} |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\Omega )-\langle \varvec{\varepsilon },\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })> \delta \right\} \\&\quad \le \sum \limits _{s = 1}^{S_n} E \left\{ ({{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\Omega )-\langle \varvec{\varepsilon },\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle )^{2N}/ (\delta \xi _n)^{2N} \right\} \\&\quad \le C (\delta \xi _n)^{-2N}\lambda _{\max }^{N}(\varvec{\Omega }) \sum \limits _{s = 1}^{S_n} \{{{\,\mathrm{\text {trace}}\,}}[\varvec{\Omega } \varvec{H}(\omega _s^0)^{T}\varvec{H}(\omega _s^0) ]\} ^{N} \\&\quad \le C^{'} (\delta \xi _n)^{-2N}\lambda _{\max }^{N}(\varvec{\Omega }) \sum \limits _{s = 1}^{S_n} R_n(\omega _s^0)^N. \end{aligned}$$

Then by combining (A.4) and second part of condition C.2, we obtain (A.2).

Note that (A.3) is equivalent to

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n}\left| \frac{||\varvec{H}(\varvec{\omega })\varvec{\varepsilon }||^{2}-{{\,\mathrm{\text {trace}}\,}}(\varvec{\Omega } \varvec{H}(\varvec{\omega })^{T} \varvec{H}(\varvec{\omega }))-2\langle A(\varvec{\omega })\varvec{\mu } ,\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle }{R_n(\varvec{\omega })}\right| . \end{aligned}$$

In order to prove (A.3), we need to show, as \(n\rightarrow \infty \),

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n}\left| \frac{||\varvec{H}(\varvec{\omega })\varvec{\varepsilon }||^{2}-{{\,\mathrm{\text {trace}}\,}}(\varvec{\Omega } \varvec{H}(\varvec{\omega })^{T} \varvec{H}(\varvec{\omega }))}{R_n(\varvec{\omega })}\right| \rightarrow 0, \end{aligned}$$


$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n}\left| \frac{\langle A(\varvec{\omega })\varvec{\mu } ,\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle }{R_n(\varvec{\omega })}\right| \rightarrow 0. \end{aligned}$$

Lemma 1 implies that \(\max \nolimits _{s = 1,\ldots , S_n}[\lambda _{\max }(\varvec{H}_{(s)})]=O(\phi (h)^{-1/2})\) and \(\max \nolimits _{s = 1,\ldots , S_n}[\lambda _{\max }(\varvec{H}_{(s)}\) \(\varvec{H}_{(s)}^T)] = O(\phi (h)^{-1})\). Then, we obtain

$$\begin{aligned}&P\left\{ \sup \limits _{\varvec{\omega } \in H_n} \left| \frac{||\varvec{H}(\varvec{\omega })\varvec{\varepsilon }||^{2}-{{\,\mathrm{\text {trace}}\,}}(\varvec{\Omega } \varvec{H}(\varvec{\omega })^{T} \varvec{H}(\varvec{\omega }))}{R_n(\varvec{\omega })}\right|> \delta \right\} \\&\quad \le P\left\{ \sup \limits _{\varvec{\omega } \in H_n} \sum \limits _{t = 1}^{S_n} \sum \limits _{s = 1}^{S_n} \omega _t \omega _s \left| \varvec{\varepsilon }^{T}\varvec{H}_{(t)}^{T}\varvec{H}_{(s)}\varvec{\varepsilon }-{{\,\mathrm{\text {trace}}\,}}(\varvec{\Omega } \varvec{H}_{(s)}^{T} \varvec{H}_{(t)})\right|> \delta \xi _n \right\} \\&\quad \le P\left\{ \max \limits _{1\le t\le S_n} \max \limits _{1\le s \le S_n}\left| \varvec{\varepsilon }^{T}\varvec{H}_{(t)}^{T}\varvec{H}_{(s)}\varvec{\varepsilon }-{{\,\mathrm{\text {trace}}\,}}(\varvec{\Omega } \varvec{H}_{(s)}^{T} \varvec{H}_{(t)})\right| > \delta \xi _n \right\} \\&\quad \le \sum \limits _{t = 1}^{S_n} \sum \limits _{s = 1}^{S_n} E\left\{ [\langle \varvec{\Omega }^{-1/2}\varvec{\varepsilon },\varvec{\Omega }^{1/2}H (\varvec{\omega }_t^0)H(\varvec{\omega }_s^0)\varvec{\Omega }^{1/2}\varvec{\Omega }^{-1/2} \varvec{\varepsilon }\rangle \right. \\&\qquad \left. -{{\,\mathrm{\text {trace}}\,}}(\varvec{\Omega } \varvec{H}(\varvec{\omega }_s^0)^{T} \varvec{H}(\varvec{\omega }_t^0))]^{2N}/(\delta \xi _n)^{2N} \right\} \\&\quad \le C \lambda _{\max }^{N}({\varvec{\Omega }})(\delta \xi _n)^{-2N} \sum \limits _{t = 1}^{S_n} \sum \limits _{s = 1}^{S_n} {{\,\mathrm{\text {trace}}\,}}( \varvec{H}(\varvec{\omega }_t^0)^{T}\varvec{H}(\varvec{\omega }_s^0) \varvec{\Omega } \varvec{H}(\varvec{\omega }_s^0)^{T} \varvec{H}(\varvec{\omega }_t^0))^{N}\\&\quad \le C S_n \lambda _{\max }^{N}({\varvec{\Omega }})(\delta \xi _n)^{-2N} \lambda _{\max }^{2N}({\varvec{H}(\varvec{\omega }_t^0)}) \sum \limits _{s = 1}^{S_n} {{\,\mathrm{\text {trace}}\,}}( \varvec{\Omega } \varvec{H}(\varvec{\omega }_s^0)^{T}\varvec{H}(\varvec{\omega }_s^0))^{N}\\&\quad \le C S_n\lambda _{\max }^{N}({\varvec{\Omega }})(\delta \xi _n)^{-2N} \phi (h)^{-N} \sum \limits _{s = 1}^{S_n} R_n({\omega _s^0})^{N}. \end{aligned}$$

And, by combining (A.4) and second part of condition C.2, we obtain (A.5). Then, from Lemma 1, we have

$$\begin{aligned}&P\left\{ \sup \limits _{\varvec{\omega } \in H_n}\left| \frac{\langle A(\varvec{\omega })\varvec{\mu } ,\varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle }{R_n(\varvec{\omega })}\right|> \delta \right\} \\&\quad \le P\left\{ \sup \limits _{\varvec{\omega } \in H_n} \sum \limits _{t = 1}^{S_n} \sum \limits _{s = 1}^{S_n} \omega _t \omega _s \left| \varvec{\varepsilon }^{T}\varvec{H}_{(t)}(I-\varvec{H}_{(s)})\varvec{\mu }\right|> \delta \xi _n \right\} \\&\quad \le P\left\{ \max \limits _{1\le t\le S_n} \max \limits _{1\le s \le S_n} \left| \varvec{\varepsilon }^{T}\varvec{H}_{(t)}(I-\varvec{H}_{(s)})\varvec{\mu }\right| > \delta \xi _n \right\} \\&\quad \le \sum \limits _{t = 1}^{S_n} \sum \limits _{s = 1}^{S_n} E\left\{ \langle \varvec{H}(\varvec{\omega }_t^0) \varvec{\varepsilon },A(\varvec{\omega }_s^0)\varvec{\mu }\rangle ^{2N}/(\delta \xi _n)^{2N} \right\} \\&\quad \le C(\delta \xi _n)^{-2N}\sum \limits _{t = 1}^{S_n} \sum \limits _{s = 1}^{S_n} || \varvec{H}(\varvec{\omega }_t^0) \varvec{\Omega }^{1/2} A(\varvec{\omega }_s^0)\varvec{\mu }||^{2N}\\&\quad \le C S_n \lambda _{\max }^{N}({\varvec{\Omega }})\lambda _{\max }^{2N}(\varvec{H}(\varvec{\omega }_t^0) ) (\delta \xi _n)^{-2N} \sum \limits _{s = 1}^{S_n} R_n(\varvec{\omega }_s^0)^{N}\\&\quad \le C S_n \lambda _{\max }^{N}({\varvec{\Omega }}) \phi (h)^{-N} (\delta \xi _n)^{-2N} \sum \limits _{s = 1}^{S_n} R_n(\varvec{\omega }_s^0)^{N}.\\ \end{aligned}$$

And, by combining (A.4) and second part of condition C.2, we have (A.6). This completes the proof of Theorem 1.

Proof of Theorem 2

Theorem 2 holds, if we have

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\hat{\varvec{\Omega }}(\varvec{\omega }))-{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })=o_p(1). \end{aligned}$$

Let \(Q_{(s)}=\text {diag}({\widetilde{h}}_{(s),11},{\widetilde{h}}_{(s),22}, \ldots ,{\widetilde{h}}_{(s),nn})\) and \(Q(\varvec{\omega }) =\sum \nolimits _{s = 1}^{S_n} w_s Q_{(s)}\), where \({\widetilde{h}}_{(s),ii}\) is the ith diagonal element of \(\varvec{H}_{(s)}\). Then, we have

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\hat{\varvec{\Omega }} (\varvec{\omega }))-{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\quad =\sup \limits _{\varvec{\omega } \in H_n}|[\varvec{Y}-\varvec{H}(\varvec{\omega })\varvec{Y}]^{T} Q(\varvec{\omega })[\varvec{Y}-\varvec{H}(\varvec{\omega })\varvec{Y}] -{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{\Omega }) |/R_n(\varvec{\omega })\\&\quad =\sup \limits _{\varvec{\omega } \in H_n}|[\varvec{\varepsilon }+\varvec{U}-\varvec{H} (\varvec{\omega })\varvec{Y}]^{T}Q(\varvec{\omega })[\varvec{\varepsilon }+\varvec{U}-\varvec{H}(\varvec{\omega }) \varvec{Y}] -{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\quad \le \sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T}Q(\varvec{\omega })\varvec{\varepsilon } -{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\qquad +\sup \limits _{\varvec{\omega } \in H_n}|(\varvec{U}-\varvec{H}(\varvec{\omega })\varvec{Y})^{T} Q(\varvec{\omega })(\varvec{U}-\varvec{H}(\varvec{\omega })\varvec{Y})|/R_n(\varvec{\omega }) \\&\qquad +2\sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T} Q(\varvec{\omega }) [\varvec{H}(\varvec{\omega })\varvec{Y}-\varvec{U}]|/R_n(\varvec{\omega })\\&\quad \le \sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T}Q(\varvec{\omega }) \varvec{\varepsilon }-{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\qquad + \sup \limits _{\varvec{\omega } \in H_n}|(\varvec{U}-\varvec{H}(\varvec{\omega })\varvec{Y})^{T} Q(\varvec{\omega })(\varvec{U}-\varvec{H}(\varvec{\omega })\varvec{Y})|/R_n(\varvec{\omega }) \\&\qquad +2\sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T} Q(\varvec{\omega }) [\varvec{H}(\varvec{\omega })\varvec{U}-\varvec{U}]|/R_n(\varvec{\omega })\\&\qquad +2\sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T} Q(\varvec{\omega }) \varvec{H}(\varvec{\omega })\varvec{\varepsilon } -{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{H} (\varvec{\omega })\varvec{\Omega } )|/R_n(\varvec{\omega })\\&\qquad +2\sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{H} (\varvec{\omega })\varvec{\Omega } )|/R_n(\varvec{\omega })\\&\quad = I_1+I_2+I_3+I_4+I_5. \end{aligned}$$

Define \(\rho =\max \limits _s\max \limits _i |{\widetilde{h}}_{(s),ii}|\). From Lemma 1, condition C.4 and condition C.5, we have

$$\begin{aligned} \rho&\le c n^{-1} \max \limits _s\{|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}_{(s)})|\}\nonumber \\&\le c n^{-1} \max \limits _s\{|{{\,\mathrm{\text {trace}}\,}}(\tilde{\varvec{H}}_{(s)})-{{\,\mathrm{\text {trace}}\,}}(\tilde{\varvec{H}}_{(s)}\varvec{K}_{(s)})|\}+c n^{-1} \max \limits _s|{{\,\mathrm{\text {trace}}\,}}(\varvec{K}_{(s)})|\nonumber \\&\le c n^{-1} \max \limits _s\{|{{\,\mathrm{\text {trace}}\,}}(\tilde{\varvec{H}}_{(s)})|\}+c n^{-1} \max \limits _s\{|{{\,\mathrm{\text {trace}}\,}}(\tilde{\varvec{H}}_{(s)}\varvec{K}_{(s)})|\}\nonumber \\&\quad +c n^{-1}\max \limits _s|{{\,\mathrm{\text {trace}}\,}}(\varvec{K}_{(s)})| \nonumber \\&= c n^{-1}{\tilde{p}}+ c n^{-1} 2^{-1}\max \limits _s\{|{{\,\mathrm{\text {trace}}\,}}(\tilde{\varvec{H}}_{(s)}\varvec{K}_{(s)}+\varvec{K}^T_{(s)}) \tilde{\varvec{H}}_{(s)})|\}\nonumber \\&\quad +c n^{-1} \max \limits _s|{{\,\mathrm{\text {trace}}\,}}(\varvec{K}_{(s)})|\nonumber \\&\le c n^{-1}{\tilde{p}}+ c n^{-1} 2^{-1}\max \limits _s\{\lambda _{\max }(\tilde{\varvec{H}}_{(s)}\varvec{K}_{(s)}+\varvec{K}^T_{(s)} \tilde{\varvec{H}}_{(s)})\text {rank}\nonumber \\&\qquad (\tilde{\varvec{H}}_{(s)}\varvec{K}_{(s)} +\varvec{K}^T_{(s)}\tilde{\varvec{H}}_{(s)})\}\nonumber \\&\quad +c n^{-1} \max \limits _s|{{\,\mathrm{\text {trace}}\,}}(\varvec{K}_{(s)})|\nonumber \\&\le c n^{-1}{\tilde{p}}+ c n^{-1} 2\max \limits _s\{p_{s}\lambda _{\max }(\tilde{\varvec{H}}_{(s)})\lambda _{\max }(\varvec{K}_{(s)})\} +c n^{-1} \max \limits _s|{{\,\mathrm{\text {trace}}\,}}(\varvec{K}_{(s)})|\nonumber \\&=O(n^{-1}{\tilde{p}}+ n^{-1}{\tilde{p}} \phi (h)^{-1/2}+n^{-1}\phi (h)^{-1}). \end{aligned}$$

From the second part of condition C.2, we have

$$\begin{aligned} \xi _n^{-1}\phi (h)^{-1}=o(1) \end{aligned}$$


$$\begin{aligned} \xi _n^{-N}\phi (h)^{-N}S_n=o(1). \end{aligned}$$

Using (A.4), Lemma 1 and (A.7), Chebyshev’s inequality and Theorem 2 of Whittle (1960), we obtain, for any \(\delta >0\),

$$\begin{aligned} P(I_1>\delta )&\le \sum \limits _{s = 1}^{S_n}P\{|\varvec{\varepsilon }^{T}Q_{(s)}\varvec{\varepsilon }-{{\,\mathrm{\text {trace}}\,}}(Q_{(s)}\varvec{\Omega })|>\delta \xi _n\}\\&\le ({\delta \xi _n})^{-2N}\sum \limits _{i = 1}^{S_n} E\{ \varvec{\varepsilon }^{T}Q_{(s)}\varvec{\varepsilon }-{{\,\mathrm{\text {trace}}\,}}(Q_{(s)}\varvec{\Omega })\}^{2N}\\&\le c ({\delta \xi _n})^{-2N}\sum \limits _{i = 1}^{S_n} {{\,\mathrm{\text {trace}}\,}}^{N} \{ \varvec{\Omega }^{1/2} Q_{(s)}\varvec{\Omega } Q_{(s)} \varvec{\Omega }^{1/2}\}\\&\le c ({\delta \xi _n})^{-2N}S_n \lambda _{\max }^{2N}(\varvec{\Omega }) \rho ^{2N} n^{N}\\&= c ({\delta \xi _n})^{-2N} S_n \lambda _{\max }^{2N}(\varvec{\Omega }) O(n^{-1}{\tilde{p}}^2+ n^{-1}{\tilde{p}}^2 \phi (h)^{-1}+n^{-1}\phi (h)^{-2})^N,\\ I_2&\le \rho \sup \limits _{\varvec{\omega } \in H_n} \{(\varvec{U}-\varvec{H}(\varvec{\omega })\varvec{Y})^{T}(\varvec{U}-\varvec{H}(\varvec{\omega })\varvec{Y})\}/R_n(\varvec{\omega })\\&=\rho \sup \limits _{\varvec{\omega } \in H_n}[L_n(\varvec{\omega })/R_n(\varvec{\omega })]=O(n^{-1}{\tilde{p}}+ n^{-1}{\tilde{p}} \phi (h)^{-1/2}+n^{-1}\phi (h)^{-1}),\\ I_3&\le 2\sup \limits _{\varvec{\omega } \in H_n} \{ ||\varvec{\varepsilon }||^2 \rho ^2 (\varvec{U}-H(\varvec{\omega })\varvec{U})^{T}(\varvec{U}-H(\varvec{\omega })\varvec{U})\}/R_n^{2}(\varvec{\omega })\}^{1/2} \\&\le 2||\varvec{\varepsilon }|| \rho \xi _n^{-1/2} = 2\xi _n^{-1/2} O(n^{-1/2}{\tilde{p}}+ n^{-1/2}{\tilde{p}} \phi (h)^{-1/2}+n^{-1/2}\phi (h)^{-1}),\\ P(I_4/2>\delta )&\le \sum \limits _{s = 1}^{S_n}P\{|\varvec{\varepsilon }^{T} Q_{(s)}\varvec{H}_{(s)}\varvec{\varepsilon } -{{\,\mathrm{\text {trace}}\,}}(Q_{(s)}\varvec{H}_{(s)}\varvec{\Omega } )|>\delta \xi _n\}\\&\le (\delta \xi _n)^{-2N}\sum \limits _{s = 1}^{S_n}E\{\varvec{\varepsilon }^{T} Q_{(s)}\varvec{H}_{(s)}\varvec{\varepsilon } -{{\,\mathrm{\text {trace}}\,}}(Q_{(s)}\varvec{H}_{(s)}\varvec{\Omega }) \}^{2N}\\&\le C (\delta \xi _n)^{-2N}\sum \limits _{s = 1}^{S_n}{{\,\mathrm{\text {trace}}\,}}^{N}\{\varvec{\Omega }^{1/2} Q_{(s)}\varvec{H}_{(s)}\varvec{\Omega } \varvec{H}_{(s)}^{T} Q_{(s)}\varvec{\Omega }^{1/2} \}\\&\le C (\delta \xi _n)^{-2N}S_n \lambda _{\max }^{2N}{(\varvec{\Omega })} n^{N} \rho ^{2N} \max \limits _s [\lambda _{\max }^N(\varvec{H}_{(s)}\varvec{H}_{(s)}^{T})] \\&= \xi _n^{-2N}\phi (h)^{-N} S_n O(n^{-1}{\tilde{p}}^2+ n^{-1}{\tilde{p}}^2 \phi (h)^{-1}+n^{-1}\phi (h)^{-2})^{N}, \\ I_5&\le 2\xi _n^{-1} \rho \lambda _{\max }(\varvec{\Omega }) \sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega }))|\\&\le 2\xi _n^{-1} \rho \lambda _{\max }(\varvec{\Omega }) \max \limits _s |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}_{(s)})|\\&= 2\xi _n^{-1} \rho \lambda _{\max }(\varvec{\Omega }) [{\tilde{p}}\phi (h)^{-1/2} + \phi (h)^{-1}]\\&= 2\xi _n^{-1}O(n^{-1}{\tilde{p}}^2\phi (h)^{-1} + n^{-1}{\tilde{p}} \phi (h)^{-3/2}+n^{-1}\phi (h)^{-2}).\\ \end{aligned}$$

Thus, by combining (A.8) and (A.9) and condition C.6, we have \(I_1+I_2+I_3+I_4+I_5=o_p(1)\). This completes the proof.

Proof of Corollary 1

Corollary 1 holds, if we can show

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\hat{\varvec{\Omega }}_{(s^{*})}) -{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega }){\varvec{\Omega }})|/R_n(\varvec{\omega })=o_p(1). \end{aligned}$$

Note that

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\hat{\varvec{\Omega }}_{(s^{*})})-{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\quad =\sup \limits _{\varvec{\omega } \in H_n}|[\varvec{Y}-\varvec{H}_{(s^{*})}\varvec{Y}]^{T}Q(\varvec{\omega })[\varvec{Y}-\varvec{H}_{(s^{*})}\varvec{Y}] -{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\quad =\sup \limits _{\varvec{\omega } \in H_n}|[\varvec{\varepsilon }+\varvec{U}-\varvec{H}_{(s^{*})}\varvec{U}-\varvec{H}_{(s^{*})} \varvec{\varepsilon }]^{T}Q(\varvec{\omega })[\varvec{\varepsilon }+\varvec{U} -\varvec{H}_{(s^{*})}\varvec{U}-\varvec{H}_{(s^{*})}\varvec{\varepsilon }]\\&\qquad -{{\,\mathrm{\text {trace}}\,}}(Q(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\quad \le \sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T}(\varvec{I}_n-\varvec{H}_{(s^{*})})^T Q(\varvec{\omega })(\varvec{I}_n-\varvec{H}_{(s^{*})})\varepsilon \\&\qquad -{{\,\mathrm{\text {trace}}\,}}((\varvec{I}_n-\varvec{H}_{(s^{*})})^{T} Q(\varvec{\omega }) (\varvec{I}_n-\varvec{H}_{(s^{*})}) \varvec{\Omega }) |/R_n(\varvec{\omega })\\&\qquad +2\sup \limits _{\varvec{\omega } \in H_n}|\varvec{\varepsilon }^{T}(\varvec{I}_n-\varvec{H}_{(s^{*})})^{T} Q(\varvec{\omega })(\varvec{I}_n-\varvec{H}_{(s^{*})})\varvec{U} |/R_n(\varvec{\omega })\\&\qquad +\sup \limits _{\varvec{\omega } \in H_n}|\varvec{U}^{T}(\varvec{I}_n-\varvec{H}_{(s^{*})})^{T} Q(\varvec{\omega })(\varvec{I}_n-\varvec{H}_{(s^{*})})\varvec{U} |/R_n(\varvec{\omega })\\&\qquad +\sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}^{T}_{(s^{*})}Q(\varvec{\omega })\varvec{H}_{(s^{*})}\varvec{\Omega } )|/R_n(\varvec{\omega })\\&\qquad +2\sup \limits _{\varvec{\omega } \in H_n}|{{\,\mathrm{\text {trace}}\,}}(\varvec{H}^{T}_{(s^{*})}Q(\varvec{\omega })\varvec{\Omega } )|/R_n(\varvec{\omega })\\&\quad = J_1+J_2+J_3+J_4+J_5. \end{aligned}$$

Following the proof (A.7) in Zhang and Wang (2019), we have \(J_1+J_2+J_3+J_4+J_5=o_p(1)\). This completes the proof.

Proof of Theorem 3

Note that

$$\begin{aligned} GCV(\varvec{\omega })&= \varvec{Y}^T\varvec{A}(\varvec{\omega })^T\varvec{A}(\varvec{\omega })\varvec{Y}+ \varvec{Y}^T \varvec{M}(\varvec{\omega })\varvec{Y}\\&= L_n(\varvec{\omega })+||\varvec{\varepsilon }||^2+ 2\varvec{\varepsilon }^T\varvec{A}(\varvec{\omega })\varvec{\mu }-2\varvec{\varepsilon }^T\varvec{H} (\varvec{\omega })\varvec{\varepsilon }\\&\quad +\varvec{\mu }^T\varvec{M}(\varvec{\omega })\varvec{\mu }+\varvec{\varepsilon }^T\varvec{M}(\varvec{\omega }) \varvec{\varepsilon }+2\varvec{\varepsilon }^T\varvec{M}(\varvec{\omega })\varvec{\mu }, \end{aligned}$$

where \(\varvec{M}(\varvec{\omega }) = \varvec{T}(\varvec{\omega })^T\varvec{A}(\varvec{\omega })+\varvec{T}(\varvec{\omega })^T\varvec{T} (\varvec{\omega })+\varvec{A}(\varvec{\omega })^T\varvec{T}(\varvec{\omega })\) and \(\varvec{T}(\varvec{\omega }) = \sum \nolimits _{s=1}^{S_n}\omega _s\varvec{D}_{(s)}\varvec{A}_{(s)}\). Theorem 3 holds if we can show that, as \(n \rightarrow \infty \),

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} | \langle \varvec{\varepsilon }, \varvec{A}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$
$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} | \langle \varvec{\varepsilon }, \varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$
$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\mu },\varvec{M}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$
$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\varepsilon },\varvec{M}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$
$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\varepsilon },\varvec{M}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$


$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |L_n(\varvec{\omega })/R_n(\varvec{\omega })-1|\rightarrow 0. \end{aligned}$$

From (A.1) and (A.3), we can obtain (A.10) and (A.15), respectively. Note that (A.11) is valid, if

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\Omega )- \langle \varvec{\varepsilon }, \varvec{H}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$


$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\Omega )|/R_n(\varvec{\omega })\rightarrow 0. \end{aligned}$$

From (A.2), we obtain (A.16). Then, from condition C.5, we have

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })&\le \xi _n^{-1} \max \limits _s |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}_{(s)}\varvec{\Omega })|\nonumber \\&= \xi _n^{-1} \max \limits _s |{{\,\mathrm{\text {trace}}\,}}({Q}_{(s)}\varvec{\Omega })|\nonumber \\&\le \lambda _{\max }(\varvec{\Omega })\xi _n^{-1} \max \limits _s \sum \limits _{i}^{n}|{\tilde{h}}_{(s),ii}|.\nonumber \\&\le \lambda _{\max }(\varvec{\Omega })\xi _n^{-1} \max \limits _s [\Lambda |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}_{(s)})|]\nonumber \\ \end{aligned}$$

From (A.7) and Lemma 1, we have

$$\begin{aligned} \max \limits _s |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}_{(s)})| = O({\widetilde{p}}\phi (h)^{-1/2} + \phi (h)^{-1}), \end{aligned}$$


$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} \lambda _{\max }(\varvec{A}(\varvec{\omega }))\le 1+\max \limits _{s \in \{1,\ldots ,S_n\}}[\lambda _{\max }(\varvec{H}_{(s)})] = O(\phi (h)^{-1/2}). \end{aligned}$$

From (A.19), condition C.7, (A.4), and (A.8), we have

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}(\varvec{\omega })\varvec{\Omega })|/R_n(\varvec{\omega })\\&\quad \le \lambda _{\max }(\varvec{\Omega })\xi _n^{-1} \max \limits _s [\Lambda |{{\,\mathrm{\text {trace}}\,}}(\varvec{H}_{(s)})|]\\&\quad = \lambda _{\max }(\varvec{\Omega })\xi _n^{-1} O({\widetilde{p}}\phi (h)^{-1/2} + \phi (h)^{-1})\\&\quad = o(1). \end{aligned}$$

Thus, equation (A.11) is true.

From (A.19) and condition C.6, we obtain \(\max \nolimits _{s \in \{1,\ldots ,S_n\}}[\lambda _{\max }(\varvec{D}_{(s)})]= {\bar{d}}/(1-{\bar{d}}) = O(\frac{ {\tilde{p}}\phi (h)^{-1/2} + \phi (h)^{-1}}{n})\), where \({\bar{d}}=\max \nolimits _{s\in \{1,\ldots ,S_n\}}[d_{(s)}]\). Then, we have

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} \lambda _{\max }(\varvec{T}(\varvec{\omega }))&\le \max \limits _{s \in \{1,\ldots ,S_n\}}[\lambda _{\max }(\varvec{D}_{(s)}\varvec{A}_{(s)})]\nonumber \\&\le \max \limits _{s\in \{1,\ldots ,S_n\}}[\lambda _{\max }(\varvec{D}_{(s)})] \max _{s \in \{1,\ldots ,S_n\}}[\lambda _{\max }(\varvec{A}_{(s)})]\nonumber \\&= O(\frac{{\tilde{p}}\phi (h)^{-1}+\phi (h)^{-3/2} }{n}). \end{aligned}$$

Equation (A.12) is valid if

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\mu },\varvec{T}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$


$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\mu },\varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\rightarrow 0. \end{aligned}$$

From conditions C.3, C.6, C.7 and (A.8), we have

$$\begin{aligned}&|\langle \varvec{\mu },\varvec{T}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\nonumber \\&\quad =| \sum \limits _{t=1}^{S_n}\sum \limits _{m=1}^{S_n} \omega _t\omega _m \varvec{\mu }^{T}\varvec{A}_{(t)}^{T}\varvec{D}_{(t)} \varvec{A}_{(m)}\varvec{\mu }|/R_n(\varvec{\omega })\nonumber \\&\quad =| \varvec{\mu }^{T}\sum \limits _{t=1}^{S_n} \omega _t \varvec{A}_{(t)}^{T}\varvec{D}_{(t)} \varvec{A}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega })\nonumber \\&\quad \le \left( \varvec{\mu }^{T}\sum \limits _{t=1}^{S_n} \omega _t \varvec{A}_{(t)}^{T}\varvec{D}_{(t)}\sum \limits _{t=1}^{S_n} \omega _t \varvec{D}_{(t)}\varvec{A}_{(t)}\varvec{\mu } \varvec{\mu }^{T}\varvec{A}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\mu }/R^2_n(\varvec{\omega })\right) ^{1/2}\nonumber \\&\quad \le \left( \varvec{\mu }^{T}\sum \limits _{t=1}^{S_n} \omega _t \varvec{A}_{(t)}^{T}\varvec{D}_{(t)}\sum \limits _{t=1}^{S_n} \omega _t \varvec{D}_{(t)}\varvec{A}_{(t)}\varvec{\mu } /R_n(\varvec{\omega })\right) ^{1/2}\nonumber \\&\quad =\left( \sum \limits _{t=1}^{S_n}\sum \limits _{m=1}^{S_n}\omega _t \omega _m \varvec{\mu }^{T}\varvec{A}_{(t)}^{T}\varvec{D}_{(t)}\varvec{D}_{(m)} \varvec{A}_{(m)}\varvec{\mu } /R_n(\varvec{\omega })\right) ^{1/2}\nonumber \\&\quad \le \xi _n^{-1}\varvec{\mu }^{T}\varvec{\mu }\lambda _{\max }(\varvec{A}_{(t)}^{T}\varvec{D}_{(t)}\varvec{D}_{(m)} \varvec{A}_{(m)})\nonumber \\&\quad = \frac{{\tilde{p}}^2\phi (h)^{-2}+\phi (h)^{-3}+{\tilde{p}}\phi (h)^{-5/2} }{n\xi _n}\nonumber \\&\quad = o(1), \end{aligned}$$

and from conditions C.3, C.6, C.7, (A.8) and (A.21), we have

$$\begin{aligned}&|\langle \varvec{\mu },\varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }\rangle |/R_n(\varvec{\omega })\nonumber \\&\quad \le \varvec{\mu }^T\varvec{\mu }\lambda _{\max }(\varvec{T}(\varvec{\omega }))\lambda _{\max }(\varvec{T}(\varvec{\omega }))/\xi _n\nonumber \\&\quad = \frac{{\tilde{p}}^2\phi (h)^{-2}+\phi (h)^{-3}+{\tilde{p}}\phi (h)^{-5/2} }{n\xi _n}\nonumber \\&\quad = o(1). \end{aligned}$$

Then, equation (A.12) is obtained. Equation (A.13) is valid if

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\varepsilon },\varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$


$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |\langle \varvec{\varepsilon },\varvec{T}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\varepsilon }\rangle |/R_n(\varvec{\omega })\rightarrow 0. \end{aligned}$$

Note that \(\langle \varvec{\varepsilon },\varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\varepsilon }\rangle \le \lambda ^2_{\max }(\varvec{T}(\varvec{\omega }))\varvec{\varepsilon }^{T}\varvec{\varepsilon }\), and from conditions C.6, C.7, (A.4), (A.8) and (A.21), we have

$$\begin{aligned}&P\left\{ \sup \limits _{\varvec{\omega } \in H_n} \lambda ^2_{\max }(\varvec{T}(\varvec{\omega }))\varvec{\varepsilon }^{T}\varvec{\varepsilon }/R_n(\varvec{\omega })>\delta \right\} \\&\quad \le P\left\{ \sup \limits _{\varvec{\omega } \in H_n} \varvec{\varepsilon }^{T}\varvec{\varepsilon }> \lambda ^{-2}_{\max }(\varvec{T}(\varvec{\omega }))\xi _n\delta \right\} \\&\quad \le E(\varvec{\varepsilon }^{T}\varvec{\varepsilon } )\lambda ^2_{\max }(\varvec{T}(\varvec{\omega })) \xi _n^{-1}\delta ^{-1}\\&\quad \le {{\,\mathrm{\text {trace}}\,}}(\Omega )\lambda ^2_{\max }(\varvec{T}(\varvec{\omega })) \xi _n^{-1}\delta ^{-1}\\&\quad \le \lambda _{\max }(\Omega )n\lambda ^2_{\max }(\varvec{T}(\varvec{\omega })) \xi _n^{-1}\delta ^{-1}\\&\quad =o(1), \end{aligned}$$

which states that (A.26) is true. Note that

$$\begin{aligned}&|\langle \varvec{\varepsilon },\varvec{T}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\varepsilon }\rangle |\nonumber \nonumber \\&\quad =|\varvec{\varepsilon }^{T}\varvec{T}^T(\varvec{\omega })\varvec{\varepsilon } - \varvec{\varepsilon }^{T}\varvec{T}^T(\varvec{\omega })\varvec{H}(\varvec{\omega })\varvec{\varepsilon }|\nonumber \nonumber \\&\quad =|\varvec{\varepsilon }^{T}\sum \limits _{s=1}^{S_n}\omega _s\varvec{D}_{(s)} \varvec{\varepsilon }-\varvec{\varepsilon }^{T}\sum \limits _{s=1}^{S_n}\omega _s\varvec{D}_{(s)}\varvec{H}_{(s)} \varvec{\varepsilon }-\varvec{\varepsilon }^{T}\varvec{T}^T(\varvec{\omega })\varvec{H}(\varvec{\omega })\varvec{\varepsilon }|\nonumber \nonumber \\&\quad \le \left| \max \limits _{s \in \{1,\ldots ,S_n\}}\lambda _{\max }(\varvec{D}_{(s)})\varvec{\varepsilon }^{T}\varvec{\varepsilon }\right| + \left| {\bar{d}}/(1-{\bar{d}})\right| \max \limits _{s \in \{1,\ldots ,S_n\}}\left| \varvec{\varepsilon }^{T}\varvec{H}_{(s)}\varvec{\varepsilon }\right| + \nonumber \nonumber \\&\qquad \qquad [(\varvec{\varepsilon }^{T}\varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\varepsilon }) (\varvec{\varepsilon }^{T}\varvec{H}(\varvec{\omega })^{T}\varvec{H}(\varvec{\omega })\varvec{\varepsilon })]^{1/2}\nonumber \nonumber \\&\quad =L_1+L_2+L_3. \end{aligned}$$

From condition C.7, \(\max \nolimits _{s \in \{1,\ldots ,S_n\}}[\lambda _{\max }(\varvec{D}_{(s)})] = O_p(\frac{ {\tilde{p}}\phi (h)^{-1/2} + \phi (h)^{-1}}{n})\), (A.4) and (A.8), we have

$$\begin{aligned} \begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} L_1/R_n(\varvec{\omega })\rightarrow 0. \end{aligned} \end{aligned}$$

Note that \(L_2 \le \left| {\bar{d}}/(1-{\bar{d}})\right| \max \nolimits _{s \in \{1,\ldots ,S_n\}}\left| \varvec{\varepsilon }^{T}\varvec{H}_{(s)}\varvec{\varepsilon } - {{\,\mathrm{\text {trace}}\,}}\{ \varvec{H}_{(s)}\varvec{\Omega }\}\right| + \left| {\bar{d}}/(1-{\bar{d}})\right| \max \nolimits _{s \in \{1,\ldots ,S_n\}}\left| {{\,\mathrm{\text {trace}}\,}}\{ \varvec{H}_{(s)}\varvec{\Omega }\}\right| \). Then, similar to (A.16) and (A.17), we have

$$\begin{aligned} \begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} L_2/R_n(\varvec{\omega })\rightarrow 0. \end{aligned} \end{aligned}$$

From (A.5) and the fact that \(R_n(\varvec{\omega })=E[L_n(\varvec{\omega })|\varvec{X},\varvec{T}]=||\varvec{H}(\varvec{\omega })\varvec{\mu }-\varvec{\mu }||^2+{{\,\mathrm{\text {trace}}\,}}\{\varvec{\Omega } \varvec{H}(\varvec{\omega })^{T}\varvec{H}(\varvec{\omega })\}>{{\,\mathrm{\text {trace}}\,}}\{\varvec{\Omega } \varvec{H}(\varvec{\omega })^{T}\varvec{H}(\varvec{\omega })\},\) we obtain

$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |\varvec{\varepsilon }^{T}\varvec{H}(\varvec{\omega })^{T}\varvec{H}(\varvec{\omega }) \varvec{\varepsilon }|/R_n(\varvec{\omega })=O_p(1). \end{aligned}$$

Combining with (A.26) and (A.29), we can show that \(\sup \nolimits _{\varvec{\omega } \in H_n} L_3/R_n(\varvec{\omega })\rightarrow 0\). Thus, equation (A.27) is true. Then, equation (A.13) is true.

Equation (A.14) is true, if

$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |\varvec{\varepsilon }^{T}\varvec{T}^{T}(\varvec{\omega })\varvec{A}(\varvec{\omega }) \varvec{\mu }|/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$
$$\begin{aligned}&\sup \limits _{\varvec{\omega } \in H_n} |\varvec{\varepsilon }^{T}\varvec{T}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega }) \varvec{\mu }|/R_n(\varvec{\omega })\rightarrow 0, \end{aligned}$$


$$\begin{aligned} \sup \limits _{\varvec{\omega } \in H_n} |\varvec{\varepsilon }^{T}\varvec{A}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega }) \varvec{\mu }|/R_n(\varvec{\omega })\rightarrow 0. \end{aligned}$$

From (A.26), the fact that \(R_n(\varvec{\omega })=E[L_n(\varvec{\omega })|\varvec{X},\varvec{T}]=||\varvec{H} (\varvec{\omega })\varvec{\mu }-\varvec{\mu }||^2+{{\,\mathrm{\text {trace}}\,}}\{\varvec{\Omega } \varvec{H}(\varvec{\omega })^{T}\varvec{H}(\varvec{\omega })\}> \varvec{\mu }^T\varvec{A}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\mu }\) and

$$\begin{aligned}&|\varvec{\varepsilon }^{T}\varvec{T}^{T}(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega })\\&\quad \le [(\varvec{\varepsilon }^{T}\varvec{T}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\varepsilon }/R_n(\varvec{\omega }))(\varvec{\mu }^T\varvec{A}^T(\varvec{\omega })\varvec{A}(\varvec{\omega })\varvec{\mu })/R_n(\varvec{\omega })]^{1/2},\\ \end{aligned}$$

we obtain (A.30). From (A.25), (A.26) and

$$\begin{aligned}&|\varvec{\varepsilon }^{T}\varvec{T}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega })\\&\quad \le [(\varvec{\varepsilon }^{T}\varvec{T}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\varepsilon }/R_n(\varvec{\omega }))(\varvec{\mu }^T\varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu })/R_n(\varvec{\omega })]^{1/2},\\ \end{aligned}$$

we obtain (A.31).

Similar to (A.1), we have \(\max \nolimits _{s\in \{ 1,\ldots , S_n\}}|\varvec{\varepsilon }^{T}\varvec{A}_{(s)}\varvec{\mu }|/R_n(\varvec{\omega }) = o_p(1)\), which along with condition C.6, (A.8), (A.25), (A.29) and

$$\begin{aligned}&|\varvec{\varepsilon }^{T}\varvec{A}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega })\\&\quad = |\varvec{\varepsilon }^{T}\varvec{T}(\varvec{\omega })\varvec{\mu }- \varvec{\varepsilon }^{T} \varvec{H}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega })\\&\quad \le |\varvec{\varepsilon }^{T}\varvec{T}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega }) + |\varvec{\varepsilon }^{T}\varvec{H}^{T}(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }|/R_n(\varvec{\omega })\\&\quad \le |{\bar{d}}/(1-{\bar{d}})|\max _{s\in \{ 1,\ldots , S_n\}}|\varvec{\varepsilon }^{T} \varvec{A}_{(s)}\varvec{\mu }|/R_n(\varvec{\omega })\\&\qquad + [(\varvec{\varepsilon }^{T}\varvec{H}^{T}(\varvec{\omega }) \varvec{H}(\varvec{\omega })\varvec{\varepsilon }/R_n(\varvec{\omega })) (\varvec{\mu }^T \varvec{T}^T(\varvec{\omega })\varvec{T}(\varvec{\omega })\varvec{\mu }/R_n(\varvec{\omega }))]^{1/2},\\ \end{aligned}$$

we obtain (A.32). Then, equation (A.14) is valid. This completes the proof.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jiang, R., Wang, L. & Bai, Y. Optimal model averaging estimator for semi-functional partially linear models. Metrika 84, 167–194 (2021). https://doi.org/10.1007/s00184-020-00772-4

Download citation


  • Semi-functional partially linear model
  • Mallows-type criterion
  • Generalized cross-validation
  • Asymptotically optimal