Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

On adaptive minimax density estimation on \(R^d\)

  • 442 Accesses

  • 25 Citations

Abstract

We address the problem of adaptive minimax density estimation on \(\mathbb{R }^d\) with \(\mathbb{L }_p\)-loss on the anisotropic Nikol’skii classes. We fully characterize behavior of the minimax risk for different relationships between regularity parameters and norm indexes in definitions of the functional class and of the risk. In particular, we show that there are four different regimes with respect to the behavior of the minimax risk. We develop a single estimator which is (nearly) optimal in order over the complete scale of the anisotropic Nikol’skii classes. Our estimation procedure is based on a data-driven selection of an estimator from a fixed family of kernel estimators.

This is a preview of subscription content, log in to check access.

References

  1. 1.

    Akakpo, N.: Adaptation to anisotropy and inhomogeneity via dyadic piecewise polynomial selection. Math. Methods Stat. 21, 1–28 (2012)

  2. 2.

    Birgé, L.: Model selection for density estimation with \({\mathbb{L}}_2\)-loss. arXiv:0808.1416v2, http://arxiv.org (2008)

  3. 3.

    Bretagnolle, J., Huber, C.: Estimation des densités: risque minimax. Z. Wahrsch. Verw. Gebiete 47, 119–137 (1979)

  4. 4.

    Delyon, B., Juditsky, A.: On minimax wavelet estimators. Appl. Comput. Harmon. Anal. 3, 215–228 (1996)

  5. 5.

    Devroye, L., Györfi, L.: Nonparametric Density Estimation The \({\mathbb{L}}_1\) View. Wiley, New York (1985)

  6. 6.

    Devroye, L., Lugosi, G.: A universally acceptable smoothing factor for kernel density estimation. Ann. Stat. 24, 2499–2512 (1996)

  7. 7.

    Devroye, L., Lugosi, G.: Nonasymptotic universal smoothing factors, kernel complexity and Yatracos classes. Ann. Stat. 25, 2626–2637 (1997)

  8. 8.

    Devroye, L., Lugosi, G.: Combinatorial Methods in Density Estimation. Springer, New York (2001)

  9. 9.

    Donoho, D.L., Johnstone, I.M., Kerkyacharian, G., Picard, D.: Density estimation by wavelet thresholding. Ann. Stat. 24, 508–539 (1996)

  10. 10.

    Efroimovich, S.Yu.: Non-parametric estimation of the density with unknown smoothness. Ann. Stat. 36, 1127–1155 (1986)

  11. 11.

    Efroimovich, S.Yu.: Adaptive estimation of and oracle inequalities for probability densities and characteristic functions. Theory Probab. Appl. 30, 557–568 (2008)

  12. 12.

    Folland, G.B.: Real Analysis, 2nd edn. Wiley, New York (1999)

  13. 13.

    Garling, D.J.H.: Inequalities: A Journey Into Linear Analysis. Cambridge University Press, Cambridge (2007)

  14. 14.

    Goldenshluger, A., Lepski, O.: Uniform bounds for norms of sums of independent random functions. Ann. Probab. 39, 2318–2384 (2011)

  15. 15.

    Goldenshluger, A., Lepski, O.: Bandwidth selection in kerrnel density estimation: oracle inequalities and adaptive minimax optimality. Ann. Stat. 39, 1608–1632 (2011)

  16. 16.

    Golubev, G.K.: Non-parametric estimation of smooth probability densities. Probl. Inform. Transm. 1, 52–62 (1992)

  17. 17.

    de Guzman, M.: Differentiation of Integrals in \(R^n\). With appendices by Antonio Crdoba, and Robert Fefferman, and two by Roberto Moriyn. Lecture Notes in Mathematics, vol. 481. Springer, Berlin (1975)

  18. 18.

    Hasminskii, R., Ibragimov, I.: On density estimation in the view of Kolmogorov’s ideas in approximation theory. Ann. Stat. 18, 999–1010 (1990)

  19. 19.

    Ibragimov, I.A., Khasminski, R.Z.: An estimate of the density of a distribution. Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOMI) 98, 61–85 (1980) (in Russian)

  20. 20.

    Ibragimov, I.A., Khasminski, R.Z.: More on estimation of the density of a distribution. Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOMI) 108, 72–88 (1981) (in Russian)

  21. 21.

    Juditsky, A., Lambert-Lacroix, S.: On minimax density estimation on \({\mathbb{R}}\). Bernoulli 10, 187–220 (2004)

  22. 22.

    Kerkyacharian, G., Picard, D., Tribouley, K.: \(L^p\) adaptive density estimation. Bernoulli 2, 229–247 (1996)

  23. 23.

    Kerkyacharian, G., Lepski, O., Picard, D.: Nonlinear estimation in anisotropic multi-index denoising. Probab. Theory Relat. Fields 121, 137–170 (2001)

  24. 24.

    Kerkyacharian, G., Lepski, O., Picard, D.: Nonlinear estimation in anisotropic multiindex denoising. Sparse case. Theory Probab. Appl. 52, 58–77 (2008)

  25. 25.

    Lepski, O.: Multivariate density estimation under sup-norm loss: oracle approach, adaptation and independence structure. Manuscript (2012)

  26. 26.

    Mason, D.M.: Risk bounds for kernel density estimators. Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOMI) 363, 66–104 (2009). http://www.pdmi.ras.ru/znsl/

  27. 27.

    Massart, P.: Concentration Inequalities and Model Selection. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6–23, 2003. Lecture Notes in Mathematics, Vol. 1896. Springer, Berlin (2007)

  28. 28.

    Nemirovski A.S.: Nonparametric estimation of smooth regression functions. Soviet J. Comput. Systems Sci. 23(6), 1–11 (1985); translated from Izv. Akad. Nauk SSSR Tekhn. Kibernet. 3, 50–60 (1985) (Russian)

  29. 29.

    Nikol’skii S.M.: Priblizhenie Funktsii Mnogikh Peremennykh i Teoremy Vlozheniya. (in Russian). [Approximation of functions of several variables and imbedding theorems.] 2nd edn, revised and supplemented. Nauka, Moscow (1977)

  30. 30.

    Reynaud-Bouret, P., Rivoirard, V., Tuleau-Malot, C.: Adaptive density estimation: a curse of support? J. Stat. Plann. Inference 141, 115–139 (2011)

  31. 31.

    Rigollet, Ph.: Adaptive density estimation using the blockwise Stein method. Bernoulli 12, 351–370 (2006)

  32. 32.

    Rigollet, Ph., Tsybakov, A.B.: Linear and convex aggregation of density estimators. Math. Methods Stat. 16, 260–280 (2007)

  33. 33.

    Samarov, A., Tsybakov, A.: Aggregation of density estimators and dimension reduction. Advances in Statistical Modeling and Inference, pp. 233–251, Ser. Biostat., Vol. 3. World Sci. Publ., Hackensack (2007)

  34. 34.

    Tsybakov, A.: Introduction to Nonparametric Estimation. Springer Series in StatisticsSpringer, New York (2009)

Download references

Author information

Correspondence to A. Goldenshluger.

Additional information

Supported by the ISF Grant No. 104/11.

Appendices

Appendix A: Proofs of auxiliary results of Sect. 5

A.1 Measurability

Write \(f(x,X^{(n)}):=\hat{f}_{\hat{h}(x)}(x)\), and note that the map \(f:\mathbb{R }^d\times \mathbb{R }^{dn}\rightarrow \mathbb{R }\) is completely determined by the kernel \(K\) and the set \(\mathcal{H }\). We need to show that \(f\) is a Borel function.

Let \(R_h(x,X^{(n)}):=\hat{R}_h(x)\), and note that for every \(h\in \mathcal{H }\), the map \(R_h:\mathbb{R }^d\times \mathbb{R }^{dn}\rightarrow \mathbb{R }\) is a continuous function. This follows from the continuity of the kernel \(K\) and from the fact that \(\mathcal{H }\) is a finite set. The continuity of \(K\) also implies that the map \(f_h:\mathbb{R }^d\times \mathbb{R }^{dn}\rightarrow \mathbb{R }\) is a continuous function for any \(h\in \mathcal{H }\), where \(f_h(x,X^{(n)}):=\hat{f}_h(x)\). Next, denote by \(\mathfrak{B }\) the Borel \(\sigma \)-algebra on \(\mathbb{R }^d\times \mathbb{R }^{dn}\), and let \(b:\mathbb{R }^d\times \mathbb{R }^{dn}\rightarrow \mathcal{H }\) be the function \(b(x,X^{(n)}):=\hat{h}(x)\). We obviously have for any given \(h\in \mathcal{H }\)

$$\begin{aligned} \left\{ (x,y)\in \mathbb{R }^d\times \mathbb{R }^{dn}: b(x,y)=h\right\}&= \bigcup _{\eta \in \mathcal{H }}\left\{ (x,y)\in \mathbb{R }^d\times \mathbb{R }^{dn}: R_h(x,y)\right. \\&\left. -R_\eta (x,y)\le 0\right\} \in \mathfrak{B }, \end{aligned}$$

where the last inclusion follows from the continuity of \(R_\eta ,\,\eta \in \mathcal{H }\). Here we have also used that \(\mathcal{H }\) is a finite set. It remains to note that

$$\begin{aligned} \hat{f}_{\hat{h}(x)}(x)=\sum _{h\in \mathcal{H }}f_h(x,X^{(n)}) \mathbf 1 \left\{ b\left( x,X^{(n)}\right) =h\right\} , \end{aligned}$$

and the required statement follows.

A.2 Proof of Lemma 1

\(1^{0}\). Note that \(\check{M}_h(g,x)=4^{-1} \hat{M}_h(g,x)\) and let \(\mathcal{H }_0=\{h\in \mathcal{H }: A_h(g,x)\ge 4\varkappa \ln n/(nV_h)\}\).

For any \(h\in \mathcal{H }_0\) we have

$$\begin{aligned} \sqrt{\frac{\varkappa A_h(g,x)\ln n}{nV_h}}\ge \frac{2\varkappa \ln n}{nV_h} \Rightarrow M_h(g,x)\le \frac{3}{4} A_h(g,x). \end{aligned}$$

Therefore,

$$\begin{aligned} |\hat{A}_h(g,x)-A_h(g,x)| \le \chi _h(g, x) + M_h(g,x)\le \chi _h(g,x) + (3/4) A_h(g,x). \end{aligned}$$

We have for any \(h\in \mathcal{H }_0\)

$$\begin{aligned} \left| \check{M}_h(g,x) - M_h(g,x)\right|&= \left| \sqrt{\frac{\varkappa \ln n}{nV_h}} \frac{\hat{A}_h(g,x)-A_h(g,x)}{\hat{A}_h^{1/2}(g,x) + A_h^{1/2}(g,x)}\right| \\&\le \sqrt{\frac{\varkappa \ln n}{nV_h}}\left( \frac{\chi _h(g,x) + (3/4) A_h(g,x)}{ A_h^{1/2}(g,x)}\right) \\&\le \frac{1}{2}\chi _h(g,x)+\frac{3}{4}M_h(g,x). \end{aligned}$$

It yields for any \(h\in \mathcal{H }_0\)

$$\begin{aligned}&\left[ \check{M}_h(g,x) \!-\! \frac{7}{4}M_h(g,x)\right] _+\!\le \! \frac{1}{2}\chi _h(g,x), \quad [M_h(g,x) \!-\! 4\check{M}_h(g,x)\quad ]_+\!\le \! 2\chi _h(g,x).\nonumber \\ \end{aligned}$$
(8.1)

(b). Now consider the set \(\mathcal{H }_1:=\mathcal{H }{\setminus }\mathcal{H }_0\). Here \(A_h(g,x)\le 4\varkappa \ln n/(nV_h)\), and, by definition of \(M_h\) we have

$$\begin{aligned} \frac{1}{4} A_h(g,x) \le \frac{\varkappa \ln n}{nV_h} \le M_h(g,x) \le \frac{3\varkappa \ln n}{nV_h},\quad \forall h\in \mathcal{H }_1. \end{aligned}$$
(8.2)

Note that we have \(\hat{M}_h(g,x)\ge \varkappa \ln n /(nV_h)\) for all \(h\). This together with (8.2) shows that

$$\begin{aligned}{}[M_h(g,x)-3 \tilde{M}_h(g,x)]_+ =0,\quad \forall h\in \mathcal{H }_1. \end{aligned}$$
(8.3)

Furthermore, for any \(h\in \mathcal{H }_1\)

$$\begin{aligned} \hat{A}_h(g,x) \le A_h(g,x)+\chi _h(g,x) + M_h(g,x) \le \chi _h(g,x) + \frac{7\varkappa \ln n}{nV_h}. \end{aligned}$$

Therefore

$$\begin{aligned} \check{M}_h(g,x)&= \sqrt{\frac{\varkappa \hat{A}_h(g,x)\ln n}{nV_h}} + \frac{\varkappa \ln n}{nV_h} \;\le \; \sqrt{\frac{\varkappa \chi _h(g,x) \ln n}{nV_h}} + (\sqrt{7}+1)\frac{\varkappa \ln n}{nV_h}\\&\le \frac{1}{2}\chi _h(g,x) + \left( \sqrt{7}+\frac{3}{2}\right) \frac{\varkappa \ln n}{nV_h} \;\le \; \frac{1}{2}\chi _h(g,x) + \left( \sqrt{7}+\frac{3}{2}\right) M_h(g,x). \end{aligned}$$

To get the penultimate inequality we have used that \(\sqrt{|ab|}\le 2^{-1}(|a|+|b|)\). Thus, it is shown that

$$\begin{aligned} \left[ \check{M}_h(g,x)- \left( \sqrt{7}+\frac{3}{2}\right) M_h(g,x)\right] _+ \le \frac{1}{2}\chi _h(g,x),\quad \forall h\in \mathcal{H }_1. \end{aligned}$$
(8.4)

Relations (8.4), (8.3) and (8.1) imply statement of the lemma. \(\square \)

A.3 Proof of Lemma 2

\(1^{0}\). Let \(g:\mathbb{R }^d\rightarrow \mathbb{R }^1\) be a fixed bounded function, and let

$$\begin{aligned} \xi _h(g, x) := \frac{1}{n}\sum _{i=1}^n g_h(X_i-x) - \int g_h(t-x)f(t)\,\mathrm{d}t,\quad h\in \mathcal{H }. \end{aligned}$$

With this notation \(\xi _h(x) = \xi _h(K,x)\) and \(\hat{A}_h(g,x)-A_h(g, x) = \xi _h(|g|, x)\). Therefore moment bounds on \(\zeta _1(x),\,\zeta _3(x)\) and \(\zeta _4(x)\) will follow from those on \(\xi _h(g,x)\) with substitution \(g \in \{ K,Q, |K|, |Q|\}\). Since \(M_h(g,x)\) depends on \(g\) only via \(|g|\) and \(\Vert g\Vert _\infty \) [see (2.6)–(2.7)], \(M_h(g, x)=M_h(|g|,x)\), and moment bounds on \(\zeta _1(x)\) and \(\zeta _3(x)\) are identical. The bound on \(\zeta _4(x)\) will follow from bounds on \(\zeta _1(x)\) and \(\zeta _3(x)\) with only one modification: kernel \(K\) should be replaced by \(Q\). As for \(\zeta _2(x)\), \(\xi _{h,\eta }(x)\) cannot be represented in terms of \(\xi _h(g,x)\) with function \(g\) independent of \(h\) and \(\eta \); see (2.3). However, the bounds on \(\zeta _2(x)\) will be obtained similarly with minor modifications. Thus it suffices to bound \(\mathbb{E }_f[\zeta _1(x)]^q\) and \(\mathbb{E }_f[\zeta _2(x)]^q\).

\(2^{0}\). We start with bounding \(\mathbb{E }_f[\zeta _1(x)]^q\). For any \(z>0,\,h\in \mathcal{H }\) and \(q\ge 1\) one has

$$\begin{aligned}&\mathbb{E }_f \left[ |\xi _h(x)| - \sqrt{\frac{2\mathrm{k}_\infty A_h(K,x)z}{nV_h}}- \frac{2\mathrm{k}_\infty z}{3nV_h}\right] _+^q \nonumber \\&\quad \le 2\Gamma (q+1) \left[ \sqrt{\frac{2\mathrm{k}_\infty A_h(K,x)}{nV_h}} + \frac{2\mathrm{k}_\infty }{3nV_h}\right] ^q e^{-z}. \end{aligned}$$
(8.5)

This inequality follows by integration of the Bernstein inequality and the following bound on the second moment of \(\xi _h(x)\):

$$\begin{aligned} \mathbb{E }_f|\xi _h(x)|^2&\le \frac{\mathrm{k}_\infty }{nV_h}\int |K_h(t-x)| f(t) \,\mathrm{d}t = \frac{\mathrm{k}_\infty A_h(K,x)}{nV_h}. \end{aligned}$$

We will show that \(\mathbb{E }_f[\zeta _1(x)]^q\) is bounded by the expression appearing on the right hand side of (5.2). In fact, we will prove a stronger inequality. Let for some \(l>0\)

$$\begin{aligned} \lambda _h:= (1+q)\ln (1/V_h) + \ln \left( F^{-1}(x) \wedge n^{l}\right) . \end{aligned}$$

In suffices to show that (5.2) holds when in the definition of \(\zeta _1(x)\) the quantity \(M_h(K,x)\) replaced by \(\tilde{M}_h(K,x)\), where

$$\begin{aligned} \tilde{M}_h(K,x)=\sqrt{\frac{2\mathrm{k}_\infty A_h(K,x)\lambda _h}{nV_h}} + \frac{2\mathrm{k}_\infty \lambda _h}{3nV_h}. \end{aligned}$$

Indeed, since and \(n^{-d}\le V_h\le 1\) for any \(h\in \mathcal{H }\), we have that

$$\begin{aligned} \lambda _h \le (q+1)d \ln n + l\ln n. \end{aligned}$$

Therefore \(\tilde{M}_h(K,x) \le M_h(K,x)\) for all \(x\in \mathbb{R }^d\) and \(h\in \mathcal{H }\) provided that

$$\begin{aligned} \varkappa \ge \mathrm{k}_\infty \left[ d(2q+4)+2l\right] . \end{aligned}$$

Thus if we establish (5.2) with \(M_h(K,x)\) replaced by \(\tilde{M}_h(K,x)\), the required bound for \(\mathbb{E }_f[\zeta _1(x)]^q\) will be proved.

We have for any \(h\in \mathcal{H }\)

$$\begin{aligned} \exp \{-\lambda _h\} = (V_h)^{q+1} \left\{ F(x)\vee n^{-l}\right\} . \end{aligned}$$

Furthermore, taking into account that \(A_h(g,x)\le V_h^{-1}\Vert g\Vert _\infty \) for any \(g\), we obtain

$$\begin{aligned} \sqrt{\frac{2\mathrm{k}_\infty A_h(K,x)}{nV_h}} + \frac{2\mathrm{k}_\infty }{3nV_h} \le \frac{2\mathrm{k}_\infty }{\sqrt{n}V_h}. \end{aligned}$$

Here we have used that \(n\ge 3\). If we set \(z=\lambda _h\) then (8.5) together with two previous display formulas yields

$$\begin{aligned} \mathbb{E }_f[\zeta _1(x)]^q&= \mathbb{E }_f \sup _{h\in \mathcal{H }} \left[ |\xi _h(x)|- M_h(K,x)\right] ^q_+ \;\le \; \sum _{h\in \mathcal{H }} \mathbb{E }_f \left[ |\xi _h(x)|- \tilde{M}_h(K,x)\right] ^q_+ \nonumber \\&\le 2\Gamma (q+1) (2\mathrm{k}_\infty )^q n^{-q/2} \left\{ F(x)\vee n^{-l}\right\} \sum _{h\in \mathcal{H }} V_h \nonumber \\&\le 2^{d+1}\Gamma (q+1)(2\mathrm{k}_\infty )^q n^{-q/2} \left\{ F(x)\vee n^{-l}\right\} . \end{aligned}$$
(8.6)

As it was mentioned above, under the same conditions inequality (8.6) holds for \(\mathbb{E }_f[\zeta _3(x)]^q\).

As for the moment bound for \(\zeta _4(x)\), in all formulas above \(K\) should be replaced by \(Q\) and \(\mathrm{k}_\infty \) by \(\mathrm{k}^2_\infty \) since \(\Vert Q\Vert _\infty \le \mathrm{k}_\infty ^2\). Specifically, if \(\varkappa \ge \mathrm{k}^2_\infty [d(2q+4)+2l]\) then

$$\begin{aligned} \mathbb{E }_f[\zeta _4(x)]^q&\le 2^{d+1}\Gamma (q+1)(2\mathrm{k}^2_\infty )^q n^{-q/2} \left\{ F(x)\vee n^{-l}\right\} \!. \end{aligned}$$

\(3^{0}\). Now we turn to bounding \(\mathbb{E }_f[\zeta _2(x)]^q\). We have similarly to (8.5)

$$\begin{aligned}&\mathbb{E }_f \left[ |\xi _{h,\eta }(x)| - \sqrt{\frac{2\mathrm{k}^2_\infty A_{h\vee \eta }(Q,x)z}{nV_{h\vee \eta }}}- \frac{2\mathrm{k}^2_\infty z}{3nV_{h\vee \eta }}\right] _+^q \nonumber \\&\quad \le \; 2\Gamma (q+1) \left[ \sqrt{\frac{2\mathrm{k}^2_\infty A_{h\vee \eta }(Q,x)}{nV_{h\vee \eta }}} + \frac{2\mathrm{k}^2_\infty }{3nV_{h\vee \eta }}\right] ^q e^{-z}. \end{aligned}$$

Here we have used the following bound on the second moment of \(\xi _{h,\eta }(x)\):

$$\begin{aligned} \mathbb{E }_f|\xi _{h,\eta }(x)|^2&\le \frac{\Vert Q_{h,\eta }\Vert _\infty }{nV_{h\vee \eta }} \int \left| \frac{1}{V_{h\vee \eta }}Q_{h,\eta } \left( \frac{t-x}{h\vee \eta }\right) \right| f(t)\,\mathrm{d}t\\&\le \frac{\Vert Q\Vert _\infty }{nV_{h\vee \eta }} \int |Q_{h\vee \eta }(t-x)| f(t)\,\mathrm{d}t = \frac{\mathrm{k}_\infty ^2 A_{h\vee \eta }(Q,x)}{nV_{h\vee \eta }}. \end{aligned}$$

The further proof goes along the same lines as the above proof with the following minor modifications: in all formulas \(\mathrm{k}_\infty \) should be replaced with \(\mathrm{k}_\infty ^2,\,V_{h\vee \eta }\) should be written instead of \(V_h\), and \(\varkappa \) should satisfy \(\varkappa \ge \mathrm{k}^2_\infty [d(2q+4)+2l]\). The statement of the lemma holds with constant \(C_0=2^{d^2+1}\Gamma (q+1)(2\mathrm{k}^2_\infty )^q\). Combining the above bounds we complete the proof. \(\square \)

Appendix B: Proofs of auxiliary results of Sect. 6

B.1. Proof of Lemma 3

We have

$$\begin{aligned}&B_h(f,x) \!=\! \int K(u) \left[ f(x+uh)\!-\!f(x)\right] \,\mathrm{d}u \!=\! \int \prod _{j=1}^d w_\ell (u_j) \left[ f(x\!+\!uh)\!-\!f(x)\right] \,\mathrm{d}u. \end{aligned}$$

First, we note that \(f(x+uh)-f(x)\) can be represented by the telescopic sum

$$\begin{aligned} f(x+uh)-f(x) \!=\! \sum _{j=1}^{d} \Delta _{u_jh_j, j} f(x_1,\ldots ,x_j, x_{j+1}+u_{j+1}h_{j+1},\ldots , x_d \!+\! u_dh_d),\nonumber \\ \end{aligned}$$
(9.1)

where we put formally \(h_{d+1}u_{d+1}=0\).

Next, for any function \(g:\mathbb{R }^d\rightarrow \mathbb{R }^1\) and \(j=1,\ldots , d\) we have

$$\begin{aligned} \int w_\ell (u_j) \Delta _{u_jh_j, j}g(x)\,\mathrm{d}u_j&= \int \sum _{i=1}^\ell \left( {\begin{array}{c}\ell \\ i\end{array}}\right) (-1)^{i+1}\frac{1}{i} w\left( \frac{u_j}{i}\right) \Delta _{u_jh_j, j}g(x)\,\mathrm{d}u_j \nonumber \\&= (-1)^{\ell -1}\int w(z) \sum _{i=1}^\ell \left( {\begin{array}{c}\ell \\ i\end{array}}\right) (-1)^{i+\ell }\Delta _{izh_j, j}g(x)\,\mathrm{d}z\nonumber \\&= (-1)^{\ell -1} \int w(z) \Delta ^\ell _{zh_j, j}\, g(x)\,\mathrm{d}z. \end{aligned}$$
(9.2)

The last equality follows from the definition of \(\ell \)-th order difference operator (3.1). Thus (9.2) and (9.1) imply that \(B_h(f,x) =\sum _{j=1}^d B_{h,j}(f,x)\), where

$$\begin{aligned}&(-1)^{\ell -1} B_{h,j}(f,x)\nonumber \\&\quad := \int \int w(z) \Delta ^\ell _{zh_j, j}\, f(x_1,\ldots ,x_j,x_{j+1}+u_{j+1}h_{j+1}, \ldots ,x_d+u_dh_d)\,\mathrm{d}z\nonumber \\&\qquad \times \prod _{m=j+1}^d w_\ell (u_m)\,\mathrm{d}u_m. \end{aligned}$$

Therefore, by the Minkowski inequality for integrals [see, e.g., [12, Section 6.3]]

$$\begin{aligned}&\left\| B_{h,j}(f, \cdot )\right\| _{r_j}\\&\quad \le \; \int \int |w(z)|\left\| \Delta ^\ell _{zh_j, j}\, f(\cdot ,\ldots ,\cdot ,\cdot +u_{j+1}h_{j+1}, \ldots ,\cdot +u_dh_d)\right\| _{r_j}\,\mathrm{d}z\\&\qquad \times \prod _{m=j+1}^d |w_\ell (u_m)|\, \mathrm{d}u_m=\int \int |w(z)|\left\| \Delta ^\ell _{zh_j, j}\, f\right\| _{r_j}\,\mathrm{d}z \prod _{m=j+1}^d |w_\ell (u_m)|\,\mathrm{d}u_m. \end{aligned}$$

Since \(f\in \mathbb{N }_{\vec {r},d}(\vec {\beta },\vec {L})\) one has

$$\begin{aligned} \left\| B_{h,j}(f, \cdot )\right\| _{r_j}\le \; L_j h_j^{\beta _j} \int \int |w(z)|\,|z|^{\beta _j}\,\mathrm{d}z \prod _{m=j+1}^d |w_\ell (u_m)|\,\mathrm{d}u_m \;\le \; C_1 L_jh_j^{\beta _j}. \end{aligned}$$

This proves (6.6). To get (6.7) we first note that the condition \(s\ge 1\) implies \(\tau (p)>0\) and \(\tau _j>0,\,j=1,\ldots ,d\). Then the inequality in (6.7) follows by the same reasoning with \(r_j\) replaced by \(q_j,\,\beta _j\) replaced by \(\gamma _j\) and with the use of embedding (6.2). \(\square \)

B.2. Proof of Proposition 1

By definition of \(J_m\) and \(\mathcal{X }_m\),

$$\begin{aligned} J_m \le 2^{p(m+1)}\varphi ^p|\mathcal{X }_m|, \end{aligned}$$
(9.3)

and now we bound from above \(|\mathcal{X }_m|\). By definition of \(\mathcal{X }_m\) we have for any \(h\in \mathcal{H }\)

$$\begin{aligned} |\mathcal{X }_m|&\le \left| \left\{ x: \sup _{\eta \ge h} M_\eta (x) >2^{m-1} \varphi \right\} \right| + \sum _{j=1}^d \left| \left\{ x: B_{h, j}^*(f,x)>2^{m-1} \varphi \right\} \right| \nonumber \\&=: J_{m,1}(h) + J_{m,2}(h). \end{aligned}$$
(9.4)

Recall that with the introduced notation

$$\begin{aligned} M_\eta (x)=\sqrt{\varkappa A_\eta (x) \delta V^{-1}_\eta } + \varkappa \delta V_\eta ^{-1}, \quad \eta \in \mathcal{H }. \end{aligned}$$

For any \(h\in \mathcal{H }\) we have

$$\begin{aligned} J_{m,2}(h)&= \sum _{j\in I{\setminus }I_\infty } \left| \{x: B_{h, j}^*(f,x)>2^{m-1} \varphi \}\right| \;+\; \sum _{j\in I_\infty } \left| \{x: B_{h, j}^*(f,x)>2^{m-1} \varphi \}\right| \nonumber \\&=: J_{m,2}^{(1)}(h) + J_{m,2}^{(2)}(h). \end{aligned}$$
(9.5)

By the Chebyshev inequality and (6.12) for any \(h\)

$$\begin{aligned}&J_{m,2}^{(1)}(h) \le \sum _{j\in I{\setminus }I_\infty } \left[ 2^{(m-1)}\varphi \right] ^{-r_j} \Vert B^{*}_{h, j}(f,\cdot )\Vert _{r_j}^{r_j} \;\le \; c_1 \sum _{j\in I{\setminus }I_\infty } 2^{-r_jm}\varphi ^{-r_j} L_j^{r_j}h_j^{\beta _jr_j}.\nonumber \\ \end{aligned}$$
(9.6)

In addition, if \(s\ge 1\) then the Chebyshev inequality and (6.13) yield

$$\begin{aligned}&J_{m,2}^{(1)}(h) \le \sum _{j\in j\in I{\setminus }I_\infty } \left[ 2^{(m-1)}\varphi \right] ^{-q_j} \Vert B^*_{h, j}(f,\cdot )\Vert _{q_j}^{q_j} \le \tilde{c}_1 \sum _{j\in I{\setminus }I_\infty } 2^{-q_jm}\varphi ^{-q_j} L_j^{q_j}h_j^{\gamma _jq_j}.\nonumber \\ \end{aligned}$$
(9.7)

In order to prove statements (i)–(iv) of the proposition we bound quantities \(J_{m,1}(h)\) and \(J_{m,2}(h)\) with bandwidth \(h=h[m]\) specified in an appropriate way.

B.2.1. Proof of statements (i) and (ii)

Note that the bound in the statement (i) coincides formally with that of the statement (ii) when \(\theta =1\). This implies that the statement (ii) in the case \(\theta =1\) follows from the statement (i). So, with slight abuse of notation, we will identify the case \(\theta =1\) with the assumption that \(f\) is a probability density.

\(1^{0}\). We start with bounding the term \(J_{m,1}(h)\) on the right hand side of (9.4). Assume that \(h\in \mathcal{H }\) is such that

$$\begin{aligned} \varkappa \delta V_h^{-1} <2^{m-2}\varphi ; \end{aligned}$$
(9.8)

then by the Chebyshev inequality

$$\begin{aligned} J_{m, 1}(h)&\le \left| \left\{ x: \sup _{\eta \ge h}\sqrt{\varkappa A_\eta (x)\delta V^{-1}_\eta } > 2^{m-2}\varphi \right\} \right| \nonumber \\&\le \sum _{\begin{array}{c} \eta \ge h\\ \eta \in \mathcal{H } \end{array}} \left| \{x: A_\eta (x) > 2^{2m-4}\varphi ^2 \varkappa ^{-1}\delta ^{-1}V_\eta \}\right| \nonumber \\&\le \left( 2^{-2m+4}\varphi ^{-2}\delta \varkappa \right) ^{\theta } \sum _{\begin{array}{c} \eta \ge h\\ \eta \in \mathcal{H } \end{array}} \Vert A_\eta \Vert ^{\theta }_\theta V_\eta ^{-\theta } \;\le \; c_2 \left( 2^{-2m}\varphi ^{-2}\delta \right) ^{\theta } \sum _{\begin{array}{c} \eta \ge h\\ \eta \in \mathcal{H } \end{array}} V_\eta ^{-\theta }, \end{aligned}$$

where we have taken into account that, for any \(\eta ,\,\Vert A_\eta \Vert _\theta \le R\) if \(f\in \mathbb{G }_\theta (R)\) with \(\theta <1\), and \(\Vert A_\eta \Vert _1\le \mathrm{k}_\infty ^2\). By definition of \(\mathcal{H }\), for any \(\eta \ge h,\,\eta \in \mathcal{H },\) we have \(V_\eta = V_h 2^{k_1+\cdots +k_d}\) for some \(k_1,\ldots ,k_d\ge 1\), which implies that \(\sum _{\eta \ge h} V_\eta ^{-\theta } \le (1-2^{-\theta })^{-d} V_h^{-\theta }\). Thus, we conclude that for any \(h\) satisfying (9.8) one has

$$\begin{aligned}&J_{m, 1}(h) \le c_3 \left( 2^{-2m}\varphi ^{-2}\delta V_h^{-1}\right) ^{\theta }. \end{aligned}$$
(9.9)

\(2^{0}\). Let \(\tilde{h}=(\tilde{h}_1,\ldots ,\tilde{h}_d)\in (0,\infty ]^d\) be given by

$$\begin{aligned}&\tilde{h}_j=\left( c_4L_j^{-1}\varphi \right) ^{1/\beta _j} 2^{\frac{m}{\beta _j}\left( 1-\frac{\theta (2+1/\beta )}{r_j(1+\theta /s)}\right) }, \quad j=1,\ldots ,d, \end{aligned}$$
(9.10)

where constant \(c_4\) will be specified later. Let us prove that \(\tilde{h}\in [n^{-1},1]^{d}\) for large enough \(n\).

Denote

$$\begin{aligned} a=\frac{1-\theta /s+1/\beta }{1+\theta /s}, \quad b_j=1-\frac{\theta (2+1/\beta )}{r_j(1+1/s)}, \end{aligned}$$

and remark that \(a>0\). We note also that

$$\begin{aligned} a^{-1} b_j=\frac{(2+1/\beta )(1-\theta /r_j)}{1-\theta /s+1/\beta }-1. \end{aligned}$$

If \(b_j\le 0\) then, because \(m\le 0\),

$$\begin{aligned} \tilde{h}_j \ge \left( c_4L_j^{-1}\varphi \right) ^{1/\beta _j} = (c_4L_j^{-1})^{1/\beta _j} (L_\beta \delta )^{\frac{1}{\beta _j(2+1/\beta )}} > n^{-1} \end{aligned}$$

for all large enough \(n\). On the other hand, since \(0\ge m\ge m_0(\theta )\) and \(2^{m_0(\theta )a} \le 2^{a}\hat{c}_1\varkappa \varphi \) by definition of \(m_0(\theta )\),

$$\begin{aligned} \tilde{h}_j \le \left( c_4L_j^{-1}\varphi \right) ^{1/\beta _j} 2^{\frac{m_0(\theta )b_j}{\beta _j}}=\left( c_4L_j^{-1}\varphi \right) ^{1/\beta _j} \left( 2^{m_0(\theta )a}\right) ^{\frac{b_j}{a\beta _j}}\\ \le (c_4L_j^{-1})^{1/\beta _j} \left( 2^{-a}\hat{c}_1\varkappa \right) ^{\frac{b_j}{a\beta _j}} \varphi ^{\frac{1+b_j/a}{\beta _j}} \le c_5 \left( c_4L_0^{-1}\right) ^{1/\beta _j}, \end{aligned}$$

where we took into account that \(1+b_j a^{-1}>0\) and \(\min _{j=1,\ldots ,d} L_j\ge L_0>0\). Then choosing constant \(c_4\) small enough we have \(\tilde{h}_j\le 1\). Thus we showed that \(\tilde{h}_j\in [n^{-1}, 1]\) for \(j\) such that \(b_j\le 0\).

Now consider the case \(b_j>0\). Here

$$\begin{aligned} \tilde{h}_j \ge (c_4L_j^{-1}\varphi )^{1/\beta _j} \left( 2^{m_0(\theta )a}\right) ^{b_ja^{-1}/\beta _j} \ge \left( c_4L_j^{-1}\varphi \right) ^{1/\beta _j}\left( \hat{c}_1\varkappa \varphi \right) ^{b_ja^{-1}/\beta _j} \nonumber \\ \ge c_6\left( c_4L_j^{-1}\right) ^{1/\beta _j} \varphi ^{\frac{(2+1/\beta )(1-\theta /r_j)}{\beta _j(1-\theta /s+1/\beta )}}~. \end{aligned}$$

It remains to note that

$$\begin{aligned} \frac{1-\theta /r_j}{\beta _j(1-\theta /s+1/\beta )} =\frac{1/\beta _j-\theta /(\beta _jr_j)}{1- \theta /s+1/\beta }<1,\quad \forall j=1,\ldots ,d, \end{aligned}$$

in view of the obvious inequality \(1/\beta -\theta /s\ge 1/\beta _j-\theta /(\beta _jr_j)\), which, in its turn, follows from the fact that \(\theta \in (0,1]\). Thus, we have that \(\tilde{h}_j>n^{-1}\) for all large enough \(n\). Furthermore, if \(b_j>0\) then since \(m\le 0\)

$$\begin{aligned} \tilde{h}_j \le \left( c_4L_j^{-1}\right) ^{1/\beta _j} \varphi ^{1/\beta _j} \le 1 \end{aligned}$$

for all large enough \(n\). Thus we have shown that \(\tilde{h}\in [n^{-1}, 1]^d\).

\(3^{0}\). Now we proceed with bounding \(J_{m,2}(h)\) for a specific choice of \(h=h[m]\), which is defined as follows. Let \(h[m]\in \mathcal{H }\) such that \(h[m]< \tilde{h}\le 2h[m]\). Let constant \(c_4\) in (9.10) be chosen so that \(c_4<(2\bar{c}_1)^{-1}\), where \(\bar{c}_1\) appears on the right hand side of (6.12). With this choice of \(c_4\) by (6.12)

$$\begin{aligned} \Vert B^*_{h[m],j}(f,\cdot )\Vert _\infty \le \bar{c}_1 L_j(h_j[m])^{\beta _j}\le \bar{c}_1 L_j\tilde{h}_j^{\beta _j}\le 2^{m-1}\varphi . \end{aligned}$$

Therefore, \(J_{m,2}^{(2)}(h[m])=0\), where \(J_{m,2}^{(2)}(\cdot )\) is defined in (9.5). Moreover, we obtain from (9.6) and \(h[m]\le \tilde{h}_j\) that

$$\begin{aligned} J_{m,2}^{(1)}(h[m])&\le c_1\sum _{j\in I{\setminus }I_\infty } 2^{-r_jm}\varphi ^{-r_j} L_j^{r_j}\tilde{h}_j^{\beta _j r_j} \le c_1\sum _{j\in I{\setminus }I_\infty } c^{r_j}_4 2^{-m \left( \frac{2+1/\beta }{1/\theta +1/s}\right) }\nonumber \\&\le c_72^{-m\left( \frac{2+1/\beta }{1/\theta +1/s}\right) }~. \end{aligned}$$
(9.11)

Note that

$$\begin{aligned} V_{h[m]}\ge 2^{-d} V_{\tilde{h}}= 2^{-d}c_4^{1/\beta } L_\beta ^{-1}\varphi ^{1/\beta }2^{m\left( \frac{1}{\beta }-\frac{2+1/\beta }{1+s/\theta }\right) }. \end{aligned}$$
(9.12)

This together with (9.9) yields

$$\begin{aligned} J_{m,1}(h[m])\le c_{8}2^{-m\left( \frac{2+1/\beta }{1/\theta +1/s}\right) }. \end{aligned}$$
(9.13)

Then it follows from (9.11) and (9.13) that

$$\begin{aligned} J_{m,1}(h[m])+J_{m,2}(h[m])\le c_{9}2^{-m\left( \frac{2+1/\beta }{1/\theta +1/s}\right) }, \end{aligned}$$

which combined with (9.3) results in

$$\begin{aligned} J_m \le c_{10} 2^{m\left( p-\frac{2+1/\beta }{1/\theta +1/s}\right) } \varphi ^p. \end{aligned}$$

Inequality (9.3) is valid only if (9.8) is fulfilled for \(h[m]\), i.e., \(\varkappa \delta V_{h[m]}^{-1} <2^{m-2}\varphi \); now we verify this condition. It is sufficient to check that \(\varkappa \delta 2^{d}V^{-1}_{\tilde{h}}< 2^{m-2}\varphi \). In view of (9.12) this inequality will follow if

$$\begin{aligned} c_4^{1/\beta }\varphi ^{1+ 1/\beta }2^{m\left( 1+\frac{1}{\beta }- \frac{2+1/\beta }{1+s/\theta }\right) }> 2^{d+2}\varkappa (L_\beta \delta ). \end{aligned}$$

Taking into account that \(L_\beta \delta =\varphi ^{2+1/\beta }\) we conclude that (9.8) is fulfilled for \(h[m]\) if

$$\begin{aligned} 2^{m\left( \frac{1-\theta /s+1/\beta }{1+\theta /s}\right) } > \hat{c}_1\varkappa \varphi , \end{aligned}$$

which is ensured by the condition \(m\ge m_0(\theta )\). This completes the proof of (6.24).

B.2.2. Proof of statement (iii)

\(1^{0}\). Let \(\hat{c}_4\) be a constant to be specified later, and let \(c_4\) be the constant given in (9.10). Let \(C_j=c_4\) if \(j\in I_\infty \) and \(C_j=\hat{c}_4\) if \(j\in I{\setminus }I_\infty \). Define \(\tilde{h}=(\tilde{h}_1,\ldots ,\tilde{h}_d)\in (0,\infty ]^d\) by the formula

$$\begin{aligned}&\tilde{h}_j=\left( C_jL_j^{-1}\varphi \right) ^{1/\beta _j} 2^{m\left( \frac{1}{\beta _j}-\frac{s(2+1/\beta )}{\beta _jr_j}\right) },\quad j=1,\ldots ,d. \end{aligned}$$
(9.14)

Note that if \(j\in I_\infty \) the corresponding coordinates of \(\tilde{h}\) given by (9.10) and (9.14) are the same.

Let us show that \(\tilde{h}\in [n^{-1},1]^d\) for large enough \(n\). First consider the coordinates \(\tilde{h}_j\) such that \(1-\frac{s}{r_j}(2+1/\beta )\ge 0\). Because \(m\ge 0\) we have for all \(n\) large enough

$$\begin{aligned} \tilde{h}_j\ge \left( C_jL_j^{-1}\varphi \right) ^{1/\beta _j}\ge \left( C_jL_j^{-1}\right) ^{1/\beta _j}(L_\beta \delta )^{ \frac{1}{2\beta _j+\beta _j/\beta }} > \delta > n^{-1}, \end{aligned}$$

where we have used the obvious inequality \(\beta _j/\beta >1\) for any \(j=1,\ldots , d\). On the other hand, because \(2^m\le \hat{c}_2\varphi ^{-1}\) we obtain

$$\begin{aligned} \tilde{h}_j&\le c_{11}\left( \hat{c}_4L_j^{-1}\right) ^{1/\beta _j}\varphi ^{ \frac{s(2+1/\beta )}{\beta _jr_j}}\rightarrow 0,\quad n\rightarrow \infty , \quad \forall j\in I{\setminus }I_\infty ;\\ \tilde{h}_j&\le c_{11}\left( C_jL_j^{-1}\right) ^{1/\beta _j}\varphi ^{\frac{s(2+1/\beta )}{\beta _jr_j}}\le c_{11}\left( c_4L_0^{-1}\right) ^{1/\beta _j}, \quad \forall j\in I_\infty . \end{aligned}$$

Thus \(\tilde{h}_j \le 1\) for large enough \(n\) if \(j\in I{\setminus }I_\infty \), and \(\tilde{h}_j\le 1\) by choice of constant \(c_4\) if \(j\in I_\infty \).

Now consider the case \(1-\frac{s}{r_j}(2+1/\beta )< 0\). Since \(2^m\le \hat{c}_2\varphi ^{-1}\)

$$\begin{aligned} \tilde{h}_j\ge c_{12}\left( C_jL_j^{-1}\right) ^{1/\beta _j}\varphi ^{\frac{s(2+1/\beta )}{\beta _jr_j}}= c_{12}\left( C_jL_j^{-1}\right) ^{1/\beta _j} (L_\beta \delta )^{\frac{s}{\beta _jr_j}}> \delta > n^{-1}, \end{aligned}$$

for all \(n\) large enough. Here we have used the obvious inequality \(1/s>1/\beta _jr_j\) \(\forall j=1,\ldots ,d\). On the other hand, since \(m\ge 0,\,\tilde{h}_j\le (C_jL_j^{-1}\varphi )^{1/\beta _j}\le 1\) for large enough \(n\). Thus we have proved that \(\tilde{h}\in [n^{-1}, 1]^d\) for all large enough \(n\).

\(2^{0}\). Let \(h[m]\in \mathcal{H }\) such that \(h[m]< \tilde{h}\le 2h[m]\) and choose constant \(c_4\) satisfies \(c_4<(2\bar{c}_1)^{-1}\) [see (6.12)]. Recall that formulas (9.10) and (9.14) coincide for \(j\in I_\infty \). Therefore, as before, with the indicated choice of \(c_4\) we have

$$\begin{aligned}&J_{m,2}^{(2)}(h[m])=0. \end{aligned}$$
(9.15)

Let \(\beta _{\pm }\) and \(\beta _\infty \) be defined by expressions \( 1/\beta _\pm :=\sum _{j\in I_+\cup I_-} 1/\beta _j\) and \(1/\beta _\infty :=\sum _{j\in I_\infty } 1/\beta _j\). We have

$$\begin{aligned}&V_{h[m]}\ge 2^{-d} V_{\tilde{h}}=2^{-d} c_4^{1/\beta _\infty }\hat{c}_4^{1/\beta _\pm } L_\beta ^{-1}\varphi ^{1/\beta }2^{-2m}. \end{aligned}$$
(9.16)

This together with \(2^m\le \hat{c}_2\varphi ^{-1}\) shows that \(V_{h[m]}\ge c_{13}\hat{c}_4^{1/\beta _\pm } L^{-1}_\beta \varphi ^{2+1/\beta }=c_{13} \hat{c}_4^{1/\beta _\pm }\delta \) and, therefore,

$$\begin{aligned} \varkappa \delta V_{h[m]}^{-1} \le c_{13}^{-1}\hat{c}_ 4^{-1/\beta _\pm }\varkappa . \end{aligned}$$
(9.17)

Remark that \(A_\eta (x) \le 2^{d}M\mathrm{k}_\infty ^2 \) for all \(x\in \mathbb{R }^d\) and \(\eta \in \mathcal{H }\). Hence, in view of (9.17)

$$\begin{aligned} \sup _{\eta \ge h[m]} M_\eta (x)&\le \mathrm{k}_\infty \sqrt{2^{d}M} \sqrt{\varkappa \delta V_{h[m]}^{-1}} + \varkappa \delta V_{h[m]}^{-1}\\&\le \left( \mathrm{k}_\infty \sqrt{2^{d}M}+\sqrt{c_{13}^{-1}\hat{c}_4^{-1/\beta _\pm }}\right) \sqrt{\delta V_{h[m]}^{-1}} =c_{14}\sqrt{\delta V_{h[m]}^{-1}}. \end{aligned}$$

It yields together with (9.16)

$$\begin{aligned} \sup _{\eta \ge \tilde{h}} M_\eta (x)\le c_{15}\,\hat{c}_4^{-1/(2\beta _\pm )}2^{m} \sqrt{(L_\beta \delta )\varphi ^{-1/\beta }}=c_{15}\, \hat{c}_4^{-1/(2\beta _\pm )} 2^{m}\varphi . \end{aligned}$$

Setting \(\hat{c}_4\) so that \(c_{15}\hat{c}_4^{-1/(2\beta _\pm )}<2^{-1}\), we obtain \(\sup _{\eta \ge \tilde{h}} M_\eta (x) \le 2^{m-1}\varphi \). This implies that

$$\begin{aligned} J_{m,1}(h[m])=0. \end{aligned}$$
(9.18)

Moreover, it follows from (9.6) and from inequality \(h[m]\le \tilde{h}\) that

$$\begin{aligned}&J_{m,2}^{(1)}(h[m])\le J_{m,2}^{(1)}(\tilde{h})\le \left[ c_1\sum _{j\in I{\setminus }I_\infty } \hat{c}^{r_j}_4\right] 2^{-ms\left( 2+1/\beta \right) }. \end{aligned}$$
(9.19)

Then (6.25) is a consequence of (9.3), (9.15), (9.18) and (9.19). The statement (ii) is proved.

B.2.3. Proof of statement (iv)

\(1^{0}\). Let \(C_j,\,j=1,\ldots , d\) be the same constants in the proof of statement (ii) in the previous section. Define \(\tilde{h}=(\tilde{h}_1,\ldots ,\tilde{h}_d)\in (0,\infty ]^d\) by the following formula

$$\begin{aligned} \tilde{h}_j&= \left( C_jL_j^{-1}\varphi \right) ^{1/\gamma _j} 2^{m\left( \frac{1}{\gamma _j}-\frac{\upsilon (2+1/\gamma )}{\gamma _jq_j}\right) } \left[ \frac{L_\gamma \varphi ^{1/\beta }}{L_\beta \varphi ^{1/\gamma }} \right] ^{\frac{\upsilon }{\gamma _jq_j}}, \quad j=1,\ldots ,d,\qquad \qquad \end{aligned}$$
(9.20)

where \(\gamma _j,\,q_j\) are defined in (6.1) and \(\gamma ,\,\upsilon \) and \(L_\gamma \) are given in (6.15).

Let us show that \(\tilde{h}\in [n^{-1},1]^d\) for large \(n\). Let \(b_j=1-\frac{\upsilon }{q_j}(2+1/\gamma )\).

First, assume that \(b_j<0\). Since \(m> 0\) and \(2^m\le \hat{c}_2\varphi ^{-1}\),

$$\begin{aligned}&\tilde{h}_j \ge c_{16}\left( C_jL_j^{-1}\right) ^{1/\gamma _j} \varphi ^{\frac{\upsilon (2+1/\gamma )}{\gamma _jq_j}} \left[ \frac{L_\gamma \varphi ^{1/\beta }}{L_\beta \varphi ^{1/\gamma }}\right] ^{\frac{\upsilon }{\gamma _jq_j}}\\&\quad = c_{16}\left( C_jL_j^{-1}\right) ^{1/\gamma _j} \left[ L_\gamma /L_\beta \right] ^{\frac{\upsilon }{\gamma _jq_j}} \varphi ^{\frac{\upsilon (2+1/\beta )}{\gamma _jq_j}}\\&\quad = c_{16}\left( C_jL_j^{-1}\right) ^{1/\gamma _j} \left[ L_\gamma \right] ^{\frac{\upsilon }{\gamma _jq_j}} \delta ^{\frac{\upsilon }{\gamma _jq_j}} >\delta >n^{-1}, \end{aligned}$$

where we have used the obvious inequality \(1/\upsilon >1/(\gamma _jq_j)\) for any \(j=1,\ldots ,d\). On the other hand, in view of \(m\ge m_1\) and by (6.20)

$$\begin{aligned} \tilde{h}_j&\le \left( C_jL_j^{-1}\varphi \right) ^{1/\gamma _j}2^{\frac{m_1b_j}{\gamma _j}} \left[ (L_\gamma /L_\beta ) \varphi ^{1/\beta -1/\gamma }\right] ^{\frac{\upsilon }{\gamma _jq_j}}\\&\le (C_jL_j^{-1}\varphi )^{1/\gamma _j}2^{\frac{m_1b_j}{\gamma _j}} \left[ 2^{m_1[\upsilon (2+1/\gamma )-s(2+1/\beta )]} \right] ^{\frac{1}{\gamma _jq_j}} \\&= \left( C_jL_j^{-1}\varphi \right) ^{1/\gamma _j} 2^{m_1\left( \frac{1}{\gamma _j}-\frac{s(2+1/\beta )}{\gamma _jq_j}\right) }. \end{aligned}$$

Then by (6.21)

$$\begin{aligned} \tilde{h}_j\le c_{17}C_1(\vec {L})C_{j}^{1/\gamma _j} \varphi ^{\frac{1}{\gamma _j}\big (1+[1-\frac{s}{q_j}{(2+1/\beta )] \frac{\upsilon (1/\beta -1/\gamma )}{\upsilon [2+1/ \gamma ]-s[2+1/\beta ]}\big )}}, \end{aligned}$$
(9.21)

where the expression for constant \(C_1(\vec {L})\) is easily found. It remains to note that

$$\begin{aligned} \upsilon (2+1/\gamma )-s(2+1/\beta )=s\upsilon \left[ (2+1/\beta )(1/s-1/\upsilon )+ (1/\gamma -1/\beta )s^{-1}\right] , \end{aligned}$$

and in view of (6.16) and (6.17)

$$\begin{aligned}&1+ \frac{[q_j-s(2+1/\beta )]\upsilon (1/\beta -1/\gamma )}{q_j[\upsilon (2+1/\gamma )-s(2+1/\beta )]}\\&\quad =\frac{2+1/\beta }{q_j} \left[ \frac{(1/s-1/\upsilon )q_j+(1/\gamma -1/\beta )}{ (1/s-1/\upsilon )(2+1/\gamma )+(1/\gamma -1/\beta )\upsilon ^{-1}} \right] >0. \end{aligned}$$

This shows that \(\tilde{h}_j \le 1\) for large \(n\).

Now assume that \(b_j\ge 0\). Then, similarly to the reasoning that resulted in (9.21) we have

$$\begin{aligned} \tilde{h}_j&\ge \left( C_jL_j^{-1}\varphi \right) ^{1/\gamma _j} 2^{\frac{m_1b_j}{\gamma _j}} \left[ (L_\gamma /L_\beta ) \varphi ^{1/\beta -1/\gamma }\right] ^{\frac{\upsilon }{\gamma _jq_j}}\nonumber \\&\ge C_1(\vec {L})C_{j}^{1/\gamma _j} \varphi ^{\frac{1}{\gamma _j}\left( 1+\left[ 1- \frac{s}{q_j}(2+1/\beta )\right] \frac{\upsilon (1/\beta -1/\gamma )}{\upsilon [2+1/\gamma ] -s[2+1/\beta ]}\right) }. \nonumber \end{aligned}$$

Since \(\varphi ^{2+1/\beta }=L_\beta \delta \),

$$\begin{aligned} \tilde{h}_j\ge c_{18}C_1(\vec {L})C_{j}^{1/\gamma _j} \delta ^{\frac{(1/s-1/\upsilon )(1/\gamma _j)+ (1/\gamma -1/\beta )(1/\gamma _j r_j)}{(1/s-1/\upsilon )(2+1/\gamma )+(1/\gamma -1/\beta ) \upsilon ^{-1}}}>\delta >n^{-1} \end{aligned}$$

for all \(n\) large enough. Here we have used (6.16), (6.17), and obvious inequalities: \(2+1/\gamma >1/\gamma _j\) and \(1/\upsilon >1/\gamma _j r_j\) for all \(j=1,\ldots , d\). On the other hand, since \(2^{m}\le \hat{c}_2\varphi ^{-1}\)

$$\begin{aligned} \tilde{h}_j&\le c_{19}\left( C_jL_j^{-1}\right) ^{1/\gamma _j} \varphi ^{\frac{\upsilon (2+1/\gamma )}{\gamma _jq_j}} \left[ \frac{L_\gamma \varphi ^{1/\beta }}{L_\beta \varphi ^{1/\gamma }} \right] ^{\frac{\upsilon }{\gamma _jq_j}} \\&= c_{19}\left( C_jL_j^{-1}\right) ^{1/\gamma _j}\left[ L_\gamma /L_ \beta \right] ^{\frac{\upsilon }{\gamma _jq_j}}\varphi ^{\frac{ \upsilon (2+1/\beta )}{\gamma _jq_j}}\\&= c_{19}\left( C_jL_j^{-1}\right) ^{1/\gamma _j}\left[ L_\gamma \right] ^{ \frac{\upsilon }{\gamma _jq_j}}\delta ^{\frac{\upsilon }{\gamma _jq_j}}. \end{aligned}$$

Therefore, \(\tilde{h}_j\rightarrow 0,\; n\rightarrow \infty ,\,\forall j\in I{\setminus }I_\infty \), and \(\tilde{h}_j\le c_{19}(c_4L_0^{-1})^{1/\gamma _j},\,\forall j\in I_\infty \). Choosing \(c_4\) small enough we come to required assertion.

\(3^{0}\). Let \(h[m]\in \mathcal{H }\) be such that \(h[m]< \tilde{h}\le 2h[m]\), and let constant \(c_4\) satisfy \(c_4<(2\bar{c}_1)^{-1}\), where \(\bar{c}_1\) is given in (6.12). With this choice of \(c_4\), if \(j\in I_\infty \) then the corresponding coordinates of \(\tilde{h}\) given by (9.14) and (9.20) coincide. Hence we have as before

$$\begin{aligned}&J_{m,2}^{(2)}(h[m])=0. \end{aligned}$$
(9.22)

Let \(\frac{1}{\gamma _\pm }:=\sum _{j\in I_+\cup I_-} \frac{1}{\gamma _j}\); then

$$\begin{aligned}&V_{h[m]}\ge 2^{-d} V_{\tilde{h}}=2^{-d}(c_4)^{1/\beta _\infty } \left( \hat{c}_4\right) ^{1/\gamma _\pm } L_\beta ^{-1}\varphi ^{1/\beta }2^{-2m}. \end{aligned}$$
(9.23)

We remark that (9.23) and (9.16) coincide up to the change in notation \(\beta _\pm \leftrightarrow \gamma _\pm \). Hence all the computations preceding (9.18) remain valid, and we have as before

$$\begin{aligned} J_{m,1}(h[m])=0. \end{aligned}$$
(9.24)

Moreover, we obtain from (9.7)

$$\begin{aligned}&J_{m,2}^{(1)}\left( h[m]\right) \le \left[ \tilde{c}_1\sum _{j\in I{\setminus }I_\infty } \hat{c}^{r_j}_4\right] \left[ \frac{L_\gamma \varphi ^{1/\beta }}{L_\beta \varphi ^{1/\gamma }} \right] ^{\upsilon }2^{-m\upsilon \left( 2+1/\gamma \right) }. \end{aligned}$$
(9.25)

The bound given in (6.26) follows now from (9.3), (9.22), (9.24) and (9.25). \(\square \)

B.3 Proof of Proposition 2

In view of (2.14) and (6.11)

$$\begin{aligned} |\hat{f}(x)-f(x)| \le c_0 [\bar{U}_f(x) + \omega (x)] \le c_1 [U_f(x) + \omega (x)] , \end{aligned}$$
(9.26)

where \(c_0\) and \(c_1\) are appropriate constants, \(\bar{U}_f(x)\) and \(U_f(x)\) are given by (6.8) and (6.10) respectively, and \(\omega (x):=\zeta (x)+\chi (x)\) with \(\zeta (x)\) and \(\chi (x)\) defined in (2.15) and (2.16).

B.3.1. Proof of statement (i)

Here for brevity we will write \(m_0=m_0(1)\). By (9.26)

$$\begin{aligned}&\int _{\mathcal{X }_{m_0}^-} |\hat{f}(x)-f(x)|^p\,\mathrm{d}x \le c_1^{p-1} \int _{\mathcal{X }_{m_0}^-} \left[ U_f(x)+\omega (x)\right] ^{p-1}|\hat{f}(x)-f(x)|\,\mathrm{d}x \nonumber \\&\quad \le \; c_2\left[ (2^{m_0}\varphi )^{p-1}\int _{\mathbb{R }^d} |\hat{f}(x)-f(x)|\,\mathrm{d}x + \int _{\mathbb{R }^d} \omega ^{p-1}(x) [2^{m_0}\varphi + \omega (x)]\,\mathrm{d}x\right] . \nonumber \end{aligned}$$

Noting that \(\Vert \hat{f}\Vert _1\le \ln ^d(n)\Vert K\Vert _1\le \ln ^d(n) \mathrm{k}_\infty \), we have \(\Vert \hat{f}-f\Vert _1\le \ln ^d(n) \mathrm{k}_\infty +1\) and, therefore,

$$\begin{aligned} (2^{m_0}\varphi )^{p-1}\int _{\mathbb{R }^d} |\hat{f}(x)-f(x)|\,\mathrm{d}x \;\le \; \left( \ln ^d(n) \mathrm{k}_\infty +1\right) (2^{m_0}\varphi )^{p-1}. \end{aligned}$$

Moreover, since \(\varkappa = \mathrm{k}_\infty ^2 [(4d+2)p+4(d+1)]\), the second statement of Theorem 1 implies

$$\begin{aligned} \mathbb{E }_f \int _{\mathbb{R }^d} \omega ^{p-1}(x) [2^{m_0}\varphi + \omega (x)]\,\mathrm{d}x \le c_3 (2^{m_0}\varphi ) n^{-(p-1)/2} + c_4 n^{-p/2}. \end{aligned}$$

Combining these inequalities and taking into account that \(2^{m_0}\varphi \le 1\) we obtain

$$\begin{aligned} J_{m_0}^-&= \mathbb{E }_f\int _{\mathcal{X }^-_{m_0}} |\hat{f}(x)\!-\!f(x)|^p\,\mathrm{d}x \!\le \! c_5\left[ \ln ^d(n)(2^{m_0}\varphi )^{p-1}\!+\!2^{m_0}\varphi n^{-(p-1)/2} \!+\! n^{-p/2}\right] \nonumber \\&\le 2c_5\left[ \ln ^d(n)(2^{m_0}\varphi )^{p-1}+ n^{-p/2}\right] .\nonumber \end{aligned}$$

By definition of \(m_0=m_0(1),\,2^{m_0}\varphi \le c_6 (L_\beta \delta )^{1/(1+1/\beta -1/s)}\); therefore

$$\begin{aligned} J_{m_0}^- \;\le \; c_7 \ln ^d(n)(L_\beta \delta )^{ \frac{p-1}{1-1/s+1/\beta }}+c_7n^{-p/2}. \end{aligned}$$

It remains to note that for large \(n\)

$$\begin{aligned} \ln ^d(n) (L_\beta \delta )^{\frac{p-1}{1+1/\beta -1/s}}\le \pi _n(L_\beta \delta )^{\nu p},\quad n^{-p/2}<(L_\beta \delta )^{\nu p}, \end{aligned}$$

and (6.27) follows.

B.3.2. Proof of statement (ii)

Let \(f^*\) be the maximal operator of \(f\) defined in (4.1). It follows from the definition of \(M_\eta (x)\) that for any \(h\in \mathcal{H }\)

$$\begin{aligned} \sup _{\eta \ge h} M_\eta (x) \le c_8\sqrt{\frac{\varkappa f^*(x)\ln n}{nV_h}} + \frac{\varkappa \ln n}{nV_h}. \end{aligned}$$

Moreover, by definition of \(\bar{B}_h(f,x),\,\bar{B}_h(f,x) \le c_{9} [f^*(x)+f(x)]\le 2c_{9} f^*(x)\) almost everywhere, where the last inequality follows from the Lebesgue differentiation theorem. Using these two inequalities and setting \(h=(1,\ldots ,1)\) in (6.8) we come to the following upper bound on \(\bar{U}_f(x)\)

$$\begin{aligned} \bar{U}_f(x)\le c_{10} \left[ f^*(x)+ \sqrt{f^*(x)\delta } + \delta \right] . \end{aligned}$$
(9.27)

In view of (6.11) we have that \(\mathcal{X }_{m_0(\theta )}^-\subseteq \mathcal{X }^-:=\{x\in \mathbb{R }^d: \bar{U}_f(x) \le \mathrm{k}_\infty 2^{m_0(\theta )}\varphi \}\); therefore if we put

$$\begin{aligned} D_1:= \mathcal{X }^- \cap \{x\in \mathbb{R }^d: f^*(x) \le \delta \},\quad D_2:=\mathcal{X }^- \cap \{x\in \mathbb{R }^d: f^*(x) > \delta \} \end{aligned}$$

then

$$\begin{aligned} J_{m_0(\theta )}^- \le \mathbb{E }_f \int _{D_1} |\hat{f}(x)-f(x)|^p\,\mathrm{d}x + \mathbb{E }_f \int _{D_2} |\hat{f}(x)-f(x)|^p\,\mathrm{d}x =: \mathbb{E }_f S_1 + \mathbb{E }_f S_2.\nonumber \\ \end{aligned}$$
(9.28)

We bound from above the two terms on the right hand side of the above inequality.

First consider \(\mathbb{E }_fS_1\). By (9.26) for any \(\theta \in (0,1]\) we have

$$\begin{aligned} S_1&= \int _{D_1} |\hat{f}(x)-f(x)|^p\,\mathrm{d}x \;\le \; c_0^{p-\theta } \int _{D_1} \left[ \bar{U}_f(x)+\omega (x)\right] ^{p-\theta }| \hat{f}(x)-f(x)|^{\theta }\,\mathrm{d}x \nonumber \\&\le c_{11}\left\{ \delta ^{p-\theta }\int _{\mathbb{R }^d} |\hat{f}(x)-f(x)|^{\theta }\,\mathrm{d}x + \int _{\mathbb{R }^d} \omega ^{p-\theta }(x) [\delta + \omega (x)]^{\theta }\,\mathrm{d}x\right\} . \nonumber \end{aligned}$$

Here we have used that, by (9.27), \(\bar{U}_f(x)\le 2 c_{10}\delta \) for all \(x\in D_1\). Remind that \(\hat{f}(x)=\hat{f}_{\hat{h}(x)}(x)\); therefore, for any \(\theta \in (0,1]\)

$$\begin{aligned}&\mathbb{E }_f|\hat{f}(x)|^{\theta }\le \left( \mathbb{E }_f|\hat{f}(x)|\right) ^{\theta } \;\le \; \left( \sum _{h\in \mathcal{H }}\mathbb{E }_f\left| \hat{f}_h(x)\right| \right) ^{\theta }\le c_{12}\left[ (\ln {n})^{d}f^*(x)\right] ^\theta . \nonumber \end{aligned}$$

Thus, for any \(f\in \mathbb{G }_\theta (R)\),

$$\begin{aligned} \delta ^{p-\theta } \mathbb{E }_f \int _{\mathbb{R }^d} |\hat{f}(x)-f(x)|^{\theta }\,\mathrm{d}x&\le \delta ^{p-\theta }\left\{ \Vert f\Vert _\theta ^\theta + c_{12}(\ln {n})^{d\theta } \Vert f^*\Vert _\theta ^\theta \right\} \\&\le c_{13} \delta ^{p-\theta }R^\theta (\ln {n})^{d\theta }. \end{aligned}$$

Furthermore, because \(\varkappa = \mathrm{k}_\infty ^2 [(4d+2)p+4(d+1)]\), by the second statement of Theorem 1

$$\begin{aligned} \mathbb{E }_f \int _{\mathbb{R }^d} \omega ^{p-\theta }(x) [\delta + \omega (x)]^{\theta }\,\mathrm{d}x \le c_{14} \delta ^{\theta } n^{-(p-\theta )/2} + c_{15} n^{-p/2}\le c_{15} n^{-p/2}. \end{aligned}$$

Combining the last two inequalities we obtain

$$\begin{aligned} \mathbb{E }_f S_1 = \mathbb{E }_f \int _{D_1} |\hat{f}(x)-f(x)|^p\,\mathrm{d}x \le c_{16} \left[ \delta ^{p-\theta }R^\theta (\ln {n})^{d\theta } + n^{-p/2}\right] . \end{aligned}$$
(9.29)

Now we proceed with bounding \(\mathbb{E }_f S_2\). We have

$$\begin{aligned}&\mathbb{E }_f S_2= \mathbb{E }_f \int _{D_2} |\hat{f}(x)-f(x)|^p\,\mathrm{d}x \;\le \; c_0^{p} \;\mathbb{E }_f \int _{D_2} \left[ \bar{U}_f(x)+\omega (x)\right] ^{p}\,\mathrm{d}x \nonumber \\&\qquad \quad \stackrel{\mathrm{(a)}}{\le } c_{17} \left[ (2^{m_0(\theta )}\varphi )^{p-\theta } \int _{D_2} |\bar{U}_f(x)|^{\theta }\,\mathrm{d}x + n^{-p/2}\right] \nonumber \\&\qquad \quad \stackrel{\mathrm{(b)}}{\le } c_{17} \left[ (2^{m_0(\theta )}\varphi )^{p-\theta }R^\theta + n^{-p/2}\right] \;\le \; c_{18} \left[ R^\theta (L_\beta \delta )^{\frac{p-\theta }{1-\theta /s+1/\beta }} + n^{-p/2}\right] .\nonumber \\ \end{aligned}$$
(9.30)

Here (a) follows from the second statement of Theorem 1 and \(\bar{U}_f(x) \le 2^{m_0(\theta )}\varphi \) for \(x\in D_2\), and (b) is valid because \(\bar{U}_f(x)\le 3c_{10}f^*(x)\) for all \(x\in D_2\), see (9.27).

Note that \(\delta ^{1-\frac{1}{1-\theta /s+1/\beta }}(\ln {n})^{\frac{d\theta }{p-\theta }}\rightarrow 0\) as \(n\rightarrow \infty \) since \(\theta \le 1\) and \(1/\beta >1/s\), where the latter inequality follows from \(r\in (1,\infty ]^d\).

Thus, combining (9.29) and (9.30) with (9.28), we obtain

$$\begin{aligned} J_{m_0(\theta )}^-\;\le \; c_{19}\left[ \left( L_\beta \delta \right) ^{\frac{p-\theta }{1-\theta /s+1/\beta }}+ n^{-p/2}\right] \le c_{20}\left( L_\beta \delta \right) ^{p\nu (\theta )}, \end{aligned}$$

as claimed. \(\square \)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Goldenshluger, A., Lepski, O. On adaptive minimax density estimation on \(R^d\) . Probab. Theory Relat. Fields 159, 479–543 (2014). https://doi.org/10.1007/s00440-013-0512-1

Download citation

Keywords

  • Density estimation
  • Oracle inequality
  • Adaptive estimation
  • Kernel estimators
  • \(\mathbb{L }_p\)-risk

Mathematics Subject Classification (2000)

  • 62G05
  • 62G20