Abstract
We consider asymptotic inference for the concentration of directional data. More precisely, we propose tests for concentration (1) in the low-dimensional case where the sample size n goes to infinity and the dimension p remains fixed, and (2) in the high-dimensional case where both n and p become arbitrarily large. To the best of our knowledge, the tests we provide are the first procedures for concentration that are valid in the (n, p)-asymptotic framework. Throughout, we consider parametric FvML tests, that are guaranteed to meet asymptotically the nominal level constraint under FvML distributions only, as well as “pseudo-FvML” versions of such tests, that meet asymptotically the nominal level constraint within the whole class of rotationally symmetric distributions. We conduct a Monte-Carlo study to check our asymptotic results and to investigate the finite-sample behavior of the proposed tests.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Amos, D.E.: Computation of modified Bessel functions and their ratios. Math. Comput. 28 (125), 239–251 (1974)
Banerjee, A., Ghosh, J.: Frequency sensitive competitive learning for scalable balanced clustering on high-dimensional hyperspheres. IEEE Trans. Neural Netw. 15, 702–719 (2004)
Banerjee, A., Dhillon, I.S., Ghosh, J., Sra, S.: Clustering on the unit hypersphere using von Mises-Fisher distributions. J. Mach. Learn. Res. 6, 1345–1382 (2005)
Briggs, M.S.: Dipole and quadrupole tests of the isotropy of gamma-ray burst locations. Astrophys. J. 407, 126–134 (1993)
Cai, T., Jiang, T.: Phase transition in limiting distributions of coherence of high-dimensional random matrices. J. Multivar. Anal. 107, 24–39 (2012)
Cai, T., Fan, J., Jiang, T.: Distributions of angles in random packing on spheres. J. Mach. Learn. Res. 14, 1837–1864 (2013)
Cutting, C., Paindaveine, D., Verdebout, T.: testing uniformity on high-dimensional spheres against monotone rotationally symmetric alternatives. Ann. Stat. (to appear)
Dryden, I.L.: Statistical analysis on high-dimensional spheres and shape spaces. Ann. Statist. 33, 1643–1665 (2005)
Fisher, R.A.: Dispersion on a sphere. Proc. R. Soc. Lond. Ser. A 217, 295–305 (1953)
Fisher, N.: Problems with the current definitions of the standard deviation of wind direction. J. Clim. Appl. Meteorol. 26 (11), 1522–1529 (1987)
Ko, D.: Robust estimation of the concentration parameter of the von Mises-Fisher distribution. Ann. Statist. 20 (2), 917–928 (1992)
Ko, D., Guttorp, P.: Robustness of estimators for directional data. Ann. Statist. 16 (2), 609–618 (1988)
Larsen, P., Blæsild, P., Sørensen, M.: Improved likelihood ratio tests on the von Mises–Fisher distribution. Biometrika 89 (4), 947–951 (2002)
Ley, C., Verdebout, T.: Local powers of optimal one-and multi-sample tests for the concentration of Fisher-von Mises-Langevin distributions. Int. Stat. Rev. 82, 440–456 (2014)
Ley, C., Paindaveine, D., Verdebout, T.: High-dimensional tests for spherical location and spiked covariance. J. Multivar. Anal. 139, 79–91 (2015)
Mardia, K.V., Jupp, P.E.: Directional Statistics, vol. 494. Wiley, New York (2009)
Paindaveine, D., Verdebout, T.: On high-dimensional sign tests. Bernoulli 22, 1745–1769 (2016)
Paindaveine, D., Verdebout, T.: Optimal rank-based tests for the location parameter of a rotationally symmetric distribution on the hypersphere. In: Hallin, M., Mason, D., Pfeifer, D., Steinebach, J. (eds.) Mathematical Statistics and Limit Theorems: Festschrift in Honor of Paul Deheuvels, pp. 249-270. Springer (2015)
Silverman, B.W.: Density Estimation for Statistics and Data Analysis, vol. 26. CRC Press, London (1986)
Stephens, M.: Multi-sample tests for the fisher distribution for directions. Biometrika 56 (1), 169–181 (1969)
Verdebout, T.: On some validity-robust tests for the homogeneity of concentrations on spheres. J. Nonparametr. Stat. 27, 372–383 (2015)
Watamori, Y., Jupp, P.E.: Improved likelihood ratio and score tests on concentration parameters of von Mises–Fisher distributions. Stat. Probabil. Lett. 72 (2), 93–102 (2005)
Watson, G.S.: Statistics on Spheres. Wiley, New York (1983)
Acknowledgements
D. Paindaveine’s research supported by an A.R.C. contract from the Communauté Française de Belgique and by the IAP research network grant P7/06 of the Belgian government (Belgian Science Policy).
T. Verdebout’s research is supported by a grant from the “Banque Nationale de Belgique”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
Proof of Theorem 1
(i) All expectations and variances when proving Part (i) of the theorem are taken under \(\mathcal{R}_{p}^{(n)}(\boldsymbol{\theta },F)\) and all stochastic convergences are taken as n → ∞ under \(\mathcal{R}_{p}^{(n)}(\boldsymbol{\theta },F)\). Since
the delta method (applied to the mapping x ↦ x∕ ∥ x ∥ ) yields
where we wrote \(\mathbf{Y}_{n}:=\bar{ \mathbf{X}}_{n}/\|\bar{\mathbf{X}}_{n}\|\). This, and the fact that
where I p denotes the p-dimensional identity matrix, readily implies that
Now, write
say. It directly follows from (5) to (7) that S 1n = o P(1) as n → ∞. As for S 2n , the central limit theorem and Slutsky’s lemma yield that S 2n is asymptotically standard normal. This readily implies that
(ii) In view of the derivations above, the continuous mapping theorem implies that, for any \(\boldsymbol{\theta }\in \mathcal{S}^{p-1}\) and \(F \in \mathcal{F}_{0}\),
as n → ∞ under \(\mathcal{R}_{p}^{(n)}(\boldsymbol{\theta },F)\). The result then follows from the fact that, under \(\mathcal{R}_{p}^{(n)}(\boldsymbol{\theta },F_{p,\kappa _{0}})\), with κ 0 = h p −1(e 10), \(\mathrm{Var}[\mathbf{X}_{1}^{{\prime}}\boldsymbol{\theta }] = 1 -\frac{p-1} {\kappa _{0}} e_{10} - e_{10}^{2};\) see, e.g., Lemma S.2.1 from [7]. □
Proof of Proposition 1
From Lemma S.2.1 in [7], we have that, under \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{p_{n},\kappa _{n}})\),
The result then readily follows from
for any ν, z > 0; see (9) in [1]. □
Proof of Theorem 2
Writing \(e_{n2}:= \mathrm{E}[(\mathbf{X}_{n1}^{{\prime}}\boldsymbol{\theta }_{n})^{2}]\), Theorem 5.1 in [7] entails that, under \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{p_{n},\kappa _{n}})\), where (κ n ) is an arbitrary sequence in (0, ∞),
converges weakly to the standard normal distribution as n → ∞. The result then follows from the fact that, under \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{p_{n},\kappa _{n}})\), where the sequence (κ n ) is such that, for any n, e n1 = e 10 under \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{p_{n},\kappa _{n}})\), one has
see Proposition 1(ii). □
The proof of Theorem 3 requires the three following preliminary results:
Lemma 1
Let Z be a random variable such that P [|Z|≤ 1] = 1. Then Var [Z 2 ] ≤ 4 Var [Z].
Lemma 2
Let the assumptions of Theorem 3 hold. Write \(\hat{e}_{n1} =\|\bar{ \mathbf{X}}_{n}\|\) and \(\hat{e}_{n2}:=\bar{ \mathbf{X}}_{n}^{{\prime}}\mathbf{S}_{n}\bar{\mathbf{X}}_{n}/\|\bar{\mathbf{X}}_{n}\|^{2}\) . Then, as n →∞ under \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{p_{n},\kappa _{n}})\) , (i) \((\hat{e}_{n1}^{2} - e_{10}^{2})/(e_{n2} - e_{10}^{2}) = o_{\mathrm{P}}(1)\) and (ii) \((\hat{e}_{2n} - e_{n2})/(e_{n2} - e_{10}^{2}) = o_{\mathrm{P}}(1)\).
Lemma 3
Let the assumptions of Theorem 3 hold. Write σ n 2 := p n (e n2 − e 10 2 ) 2 + 2np n e 10 2 (e n2 − e 10 2 ) + (1 − e n2 ) 2 and \(\hat{\sigma }_{n}^{2}:= p_{n}(\hat{e}_{n2} -\hat{ e}_{n1}^{2})^{2} + 2np_{n}e_{10}^{2}(\hat{e}_{n2} -\hat{ e}_{n1}^{2}) + (1 -\hat{ e}_{n2})^{2}\) . Then \((\hat{\sigma }_{n}^{2} -\sigma _{n}^{2})/\sigma _{n}^{2} = o_{\mathrm{P}}(1)\) as n →∞ under \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{p_{n},\kappa _{n}})\).
Proof of Lemma 1
Let Z a and Z b be mutually independent and identically distributed with the same distribution as Z. Since | x 2 − y 2 | ≤ 2 | x − y | for any x, y ∈ [−1, 1], we have that
which proves the result. □
Proof of Lemma 2
All expectations and variances in this proof are taken under the sequence of hypotheses \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{n})\) considered in the statement of Theorem 3, and all stochastic convergences are taken as n → ∞ under the same sequence of hypotheses. (i) Proposition 5.1 from [7] then yields
and
as n → ∞. In view of Condition (i) in Theorem 3, this readily implies
as n → ∞, which establishes Part (i) of the result.
(ii) Write
Part (i) of the result shows that \((\hat{e}_{n1}^{2} - e_{10}^{2})/\tilde{e}_{n2}\) is o P(1) as n → ∞. Since (10) and (11) yield that \(\hat{e}_{n1}\) converges in probability to e 10( ≠ 0), this implies that \((\hat{e}_{n1}^{-2} - e_{10}^{-2})/\tilde{e}_{n2}\) is o P(1) as n → ∞. This, and the fact that \(\bar{\mathbf{X}}_{n}^{{\prime}}\mathbf{S}_{n}\bar{\mathbf{X}}_{n} = O_{\mathrm{P}}(1)\) as n → ∞, readily yields
as n → ∞. Since
the result follows if we can prove that
all are o P(1) as n → ∞.
Starting with A n , (10) yields
as n → ∞. Since convergence in L 1 is stronger than convergence in probability, this implies that A n = o P(1) as n → ∞. Turning to B n , the Cauchy–Schwarz inequality and (13) provide
as n → ∞, so that B n is indeed o P(1) as n → ∞. Finally, it follows from Lemma 1 that
as n → ∞, so that C n is also o P(1) as n → ∞. This establishes the result. □
Proof of Lemma 3
As in the proof of Lemma 2, all expectations and variances in this proof are taken under the sequence of hypotheses \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{n})\) considered in the statement of Theorem 3, and all stochastic convergences are taken as n → ∞ under the same sequence of hypotheses.
Let then \(\tilde{\sigma }_{n}^{2}:= 2np_{n}e_{10}^{2}(e_{n2} - e_{10}^{2})\). Since Condition (i) in Theorem 3 directly entails that \(\sigma _{n}^{2}/\tilde{\sigma }_{n}^{2} \rightarrow 1\) as n → ∞, it is sufficient to show that \((\hat{\sigma }_{n}^{2} -\sigma _{n}^{2})/\tilde{\sigma }_{n}^{2}\) is o P(1) as n → ∞. To do so, write
where
and
Since
almost surely, Condition (i) in Theorem 3 implies that \(A_{n}/\tilde{\sigma }_{n}^{2}\) and \(C_{n}/\tilde{\sigma }_{n}^{2}\) are o P(1) as n → ∞. The result then follows from the fact that, in view of Lemma 2,
is also o P(1) as n → ∞. □
Proof of Theorem 3
Decompose Q CPVm (n) into
say. Theorem 5.1 in [7] entails that, under the sequence of hypotheses \(\mathcal{R}_{p_{n}}^{(n)}(\boldsymbol{\theta }_{n},F_{n})\) considered in the statement of the theorem, V n is asymptotically standard normal as n → ∞. The result therefore follows from Lemma 3 and the Slutsky’s lemma. □
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Cutting, C., Paindaveine, D., Verdebout, T. (2017). Tests of Concentration for Low-Dimensional and High-Dimensional Directional Data. In: Ahmed, S. (eds) Big and Complex Data Analysis. Contributions to Statistics. Springer, Cham. https://doi.org/10.1007/978-3-319-41573-4_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-41573-4_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-41572-7
Online ISBN: 978-3-319-41573-4
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)