Kernel Tests for One, Two, and K-Sample Goodness-of-Fit: State of the Art and Implementation Considerations

  • Yang Chen
  • Marianthi MarkatouEmail author
Part of the Emerging Topics in Statistics and Biostatistics book series (ETSB)


In this article, we first discuss the fundamental role of statistical distances in the problem of goodness-of-fit and review various existing multivariate two-sample goodness-of-fit tests from both statistics and machine learning literature. The review conducted delivers the fact that there does not exist a satisfactory multivariate two-sample goodness-of-fit test. We introduce a class of one and two-sample tests constructed using the kernel-based quadratic distance, and briefly touch upon their asymptotic properties. We discuss the practical implementation of these tests, with emphasis on the kernel-based two-sample test. Finally, we use simulations and real data to illustrate the application of the kernel-based two-sample test, and compare this test with tests existing in the literature.


Goodness-of-fit Kernel tests Quadratic distance Multivariate Two-sample methods 



The work of both authors is supported by The Troup Fund of the Kaleida Health Foundation.


  1. 1.
    Anderson, N. H., Hall, P., & Titterington, D. M. (1994). Two-sample test statistics for measuring discrepancies between two multivariate probability density functions using kernel-based density estimates. Journal of Multivariate Analysis, 50(1), 41–54.MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Aslan, B., & Zech, G. (2005). Statistical energy as a tool for binning-free, multivariate goodness-of-fit tests, two-sample comparison and unfolding. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 537(3), 626–636.CrossRefGoogle Scholar
  3. 3.
    Barakat, A. S., Quade, D., & Salama, I. A. (1996). Multivariate homogeneity testing using an extended concept of nearest neighbors. Biometrical Journal, 38(5), 605–612.zbMATHCrossRefGoogle Scholar
  4. 4.
    Baringhaus, L., & Franz, C. (2004). On a new multivariate two-sample test. Journal of Multivariate Analysis, 88(1), 190–206.MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Bickel, P. J. (1969). A distribution free version of the smirnov two sample test in the p-variate case. The Annals of Mathematical Statistics, 40(1), 1–23.MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Biswas, M., & Ghosh, A. K. (2014). A nonparametric two-sample test applicable to high dimensional data. Journal of Multivariate Analysis, 123(1), 160–171.MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Biswas, M., Mukhopadhyay, M., & Ghosh, A. K. (2014). A distribution-free two-sample run test applicable to high-dimensional data. Biometrika, 101(4), 913–926.MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Blaug, M. (1980). The methodology of economics: Or, how economists explain. Cambridge: Cambridge University Press.Google Scholar
  9. 9.
    Burke, M. D. (2000). Multivariate tests-of-fit and uniform confidence bands using a weighted bootstrap. Statistics and Probability Letters, 46(1), 13–20.MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Cao, R., & Lugosi, G. (2005). Goodness-of-fit tests based on the kernel density estimator. Scandinavian Journal of Statistics, 32(4), 599–616.MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Cao, R., & Van Keilegom, I. (2006). Empirical likelihood tests for two-sample problems via nonparametric density estimation. The Canadian Journal of Statistics, 1(34), 61–77.MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Chen, H., & Friedman, J. H. (2017). A new graph-based two-sample test for multivariate and object data. Journal of the American statistical association, 112(517), 397–409.MathSciNetCrossRefGoogle Scholar
  13. 13.
    Chen, L., Dou, W. W., & Qiao, Z. (2013). Ensemble subsampling for imbalanced multivariate two-sample tests. Journal of the American Statistical Association, 108(504), 1308–1323.MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    D’Agostino, R. B., & Stephen, M. A. (1986). Goodness-of-fit techniques. New York: Marcel Dekker.zbMATHGoogle Scholar
  15. 15.
    Danafar, S., Rancoita, P., Glasmachers, T., Whittingstall, K., & Schmidhuber, J. (2014). Testing hypotheses by regularized maximum mean discrepancy. International Journal of Computer and Information Technology, 3(2), 223–232.Google Scholar
  16. 16.
    Durbin, J. (1973). Distribution theory for tests based on the sample distribution function. CBMS-NSF, Regional Conference Series in Applied Mathematics. Philadelphia: SIAM.Google Scholar
  17. 17.
    Eric, M., Bach, F. R., & Harchaoui, Z. (2008). Testing for homogeneity with kernel fisher discriminant analysis. In Advances in neural information processing systems (Vol. 20, pp. 609–616).Google Scholar
  18. 18.
    Fernández, V. A., Gamero, M. J., & García, J. M. (2008). A test for the two-sample problem based on empirical characteristic functions. Computational Statistics & Data Analysis, 52(7), 3730–3748.MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Friedman, J. H., & Rafsky, L. C. (1979). Multivariate generalizations of the Wald-Wolfowitz and Smirnov two-sample tests. The Annals of Statistics, 7(4), 697–717.MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B., & Smola, A. (2012a). A kernel two-sample test. The Journal of Machine Learning Research, 13(1), 723–773.MathSciNetzbMATHGoogle Scholar
  21. 21.
    Gretton, A., Sejdinovic, D., Strathmann, H., Balakrishnan, S., Pontil, M., Fukumizu, K., & Sriperumbudur, B. K. (2012b). Optimal kernel choice for large-scale two-sample tests. Advances in Neural Information Processing Systems, 25, 1214–1222.Google Scholar
  22. 22.
    Hall, P., & Tajvidi, N. (2002). Permutation tests for equality of distributions in high dimensional settings. Biometrika, 89(2), 359–374.MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Henze, N. (1984). On the number of random points with nearest neighbors of the same type and a multivariate two-sample test. Metrika, 31, 259–273.MathSciNetCrossRefGoogle Scholar
  24. 24.
    Henze, N. (1988). A multivariate two-sample test based on the number of nearest neighbor type coincidences. The Annals of Statistics, 16(2), 772–783.MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Huo, X., & Székely, G. J. (2016). Fast computing for distance covariance. Technometrics, 58(4), 435–447.MathSciNetCrossRefGoogle Scholar
  26. 26.
    Karoui, N. E., & Purdom, E. (2016). Can we trust the bootstrap in high-dimension? arXiv:1608.00696.Google Scholar
  27. 27.
    Lindsay, B. G., Markatou, M., & Ray, S. (2014). Kernels, degrees of freedom, and power properties of quadratic distance goodness-of-fit tests. Journal of the American Statistical Association, 109(505), 395–410.MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    Lindsay, B. G., Markatou, M., Ray, S., Yang, K., & Chen, S. C. (2008). Quadratic distances on probabilities: A unified foundation. The Annals of Statistics, 36(2), 983–1006.MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Liu, Z., & Modarres, R. (2011). A triangle test for equality of distribution functions in high dimensions. Journal of Nonparametric Statistics, 23(3), 605–615.MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Lyons, R. (2013). Distance covariance in metrics spaces. The Annals of Statistics, 41(5), 3284–3305.MathSciNetzbMATHGoogle Scholar
  31. 31.
    Maag, U. R., & Stephens, M. A. (1968). The v nm two-sample test. The Annals of Mathematical Statistics, 39(3), 923–935.MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Mondal, P. K., Biswas, M., & Ghosh, A. K. (2015). On high dimensional two-sample tests based on nearest neighbors. Journal of Multivariate Analysis, 141, 168–178.MathSciNetzbMATHCrossRefGoogle Scholar
  33. 33.
    Muandet, K., Fukumizu, K., Sriperumbudur, B., & Schölkopf, B. (2017). Kernel mean embedding of distributions: A review and beyond. Foundations and Trends in Machine Learning, 10(1), 1–141.zbMATHCrossRefGoogle Scholar
  34. 34.
    Neuhaus, G. (1977). Functional limit theorems for u-statistics in the degenerate case. Journal of Multivariate Analysis, 7(3), 424–439.MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    Pettitt, A. N. (1976a). A two-sample anderson-darling rank statistic. Biometrika, 63(1), 161–168.MathSciNetzbMATHGoogle Scholar
  36. 36.
    Pettitt, A. N. (1976b). Two-sample cramér-von mises type rank statistics. Journal of the Royal Statistical Society: Series B, 41(1), 46–53.zbMATHGoogle Scholar
  37. 37.
    Póczos, B., Ghahramani, Z., & Schneider, J. (2012). Copula-based kernel dependency measures. Proceedings of the 29th International Conference on Machine Learning (pp. 775–782).Google Scholar
  38. 38.
    Popper, K. R. (1986). The logic of scientific discovery (2nd ed.). New York: Harper and Row.zbMATHGoogle Scholar
  39. 39.
    Rayner, J. C. W., & Best, D. J. (1986). Smooth tests of goodness of fit. Oxford: Oxford University Press.zbMATHGoogle Scholar
  40. 40.
    Reddi, S., Ramdas, A., Póczos, B., Singh, A., & Wasserman, L. (2015). On the decreasing power of kernel and distance based nonparametric hypothesis tests in high dimensions. Proceedings of 29th AAAI Conference on Artificial Intelligence (pp. 3571–3577).Google Scholar
  41. 41.
    Rosenbaum, P. R. (2005). An exact distribution free test comparing two multivariate distributions based on adjacency. Journal of the Royal Statistical Society: Series B, 67(4), 515–530.MathSciNetzbMATHCrossRefGoogle Scholar
  42. 42.
    Savage, L. T. (1972). The foundations of statistics (2nd ed.). New York: Dover.zbMATHGoogle Scholar
  43. 43.
    Schilling, M. F. (1986). Multivariate two-sample tests based on nearest neighbors. Journal of the American Statistical Association, 81(395), 799–806.MathSciNetzbMATHCrossRefGoogle Scholar
  44. 44.
    Sejdinovic, D., Surperumbudur, B., Gretton, A., & Fukumizu, K. (2013). Equivalence of distance-based and RKHS-based statistics in hypothesis testing. The Annals of Statistics, 41(3), 2263–2291.MathSciNetzbMATHCrossRefGoogle Scholar
  45. 45.
    Szabo, A., Boucher, K., Carroll, W. L., Klebanov, L. B., Tsodikov, A. D., & Yakovlev, A. Y. (2002). Variable selection and pattern recognition with gene expression data generated by the microarray technology. Mathematical Biosciences, 176(1), 71–98.MathSciNetzbMATHCrossRefGoogle Scholar
  46. 46.
    Szabo, A., Boucher, K., Jones, D., Tsodikov, A. D., Klebanov, L. B., & Yakovlev, A. Y. (2003). Multivariate exploratory tools for microarray data analysis. Biostatistics, 4(4), 555–567.zbMATHCrossRefGoogle Scholar
  47. 47.
    Székely, G. J., & Rizzo, M. L. (2004). Testing for equal distributions in high dimension. InterStat, 5(1), 1–6.Google Scholar
  48. 48.
    Székely, G. J., & Rizzo, M. L. (2005). A new test for multivariate normality. Journal of Multivariate Analysis, 93(1), 58–80.MathSciNetzbMATHCrossRefGoogle Scholar
  49. 49.
    Székely, G. J., & Rizzo, M. L. (2013). Energy statistics: A class of statistics based on distances. Journal of Statistical Planning and Inference, 143(8), 1249–1272.MathSciNetzbMATHCrossRefGoogle Scholar
  50. 50.
    Wald, A. (1950). Statistical decision functions. New York: John Wiley & Sons.zbMATHGoogle Scholar
  51. 51.
    Wald, A., & Wolfowitz, J. (1940). On a test whether two samples are from the same population. The Annals of Mathematical Statistics, 11(2), 147–162.MathSciNetzbMATHCrossRefGoogle Scholar
  52. 52.
    Wu, M. C., Lee, S., Cai, T., Li, Y., Boehnke, M., & Lin, X. (2011). Rare-variant association testing for sequencing data with the sequence kernel association test. The American Journal of Human Genetics, 89(1), 82–93.CrossRefGoogle Scholar
  53. 53.
    Zaremba, W., Gretton, A., & Blaschko, M. (2013). B-test: A non-parametric, low variance kernel two-sample test. Advances in neural information processing systems (pp. 755–763)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of BiostatisticsUniversity at BuffaloBuffaloUSA

Personalised recommendations