Skip to main content
SpringerLink
Log in

Kernel Tests for One, Two, and K-Sample Goodness-of-Fit: State of the Art and Implementation Considerations

  • Chapter
  • First Online:
Statistical Modeling in Biomedical Research

Part of the book series: Emerging Topics in Statistics and Biostatistics ((ETSB))

Abstract

In this article, we first discuss the fundamental role of statistical distances in the problem of goodness-of-fit and review various existing multivariate two-sample goodness-of-fit tests from both statistics and machine learning literature. The review conducted delivers the fact that there does not exist a satisfactory multivariate two-sample goodness-of-fit test. We introduce a class of one and two-sample tests constructed using the kernel-based quadratic distance, and briefly touch upon their asymptotic properties. We discuss the practical implementation of these tests, with emphasis on the kernel-based two-sample test. Finally, we use simulations and real data to illustrate the application of the kernel-based two-sample test, and compare this test with tests existing in the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 74.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Anderson, N. H., Hall, P., & Titterington, D. M. (1994). Two-sample test statistics for measuring discrepancies between two multivariate probability density functions using kernel-based density estimates. Journal of Multivariate Analysis, 50(1), 41–54.

    Article  MathSciNet  MATH  Google Scholar 

  2. Aslan, B., & Zech, G. (2005). Statistical energy as a tool for binning-free, multivariate goodness-of-fit tests, two-sample comparison and unfolding. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 537(3), 626–636.

    Article  Google Scholar 

  3. Barakat, A. S., Quade, D., & Salama, I. A. (1996). Multivariate homogeneity testing using an extended concept of nearest neighbors. Biometrical Journal, 38(5), 605–612.

    Article  MATH  Google Scholar 

  4. Baringhaus, L., & Franz, C. (2004). On a new multivariate two-sample test. Journal of Multivariate Analysis, 88(1), 190–206.

    Article  MathSciNet  MATH  Google Scholar 

  5. Bickel, P. J. (1969). A distribution free version of the smirnov two sample test in the p-variate case. The Annals of Mathematical Statistics, 40(1), 1–23.

    Article  MathSciNet  MATH  Google Scholar 

  6. Biswas, M., & Ghosh, A. K. (2014). A nonparametric two-sample test applicable to high dimensional data. Journal of Multivariate Analysis, 123(1), 160–171.

    Article  MathSciNet  MATH  Google Scholar 

  7. Biswas, M., Mukhopadhyay, M., & Ghosh, A. K. (2014). A distribution-free two-sample run test applicable to high-dimensional data. Biometrika, 101(4), 913–926.

    Article  MathSciNet  MATH  Google Scholar 

  8. Blaug, M. (1980). The methodology of economics: Or, how economists explain. Cambridge: Cambridge University Press.

    Google Scholar 

  9. Burke, M. D. (2000). Multivariate tests-of-fit and uniform confidence bands using a weighted bootstrap. Statistics and Probability Letters, 46(1), 13–20.

    Article  MathSciNet  MATH  Google Scholar 

  10. Cao, R., & Lugosi, G. (2005). Goodness-of-fit tests based on the kernel density estimator. Scandinavian Journal of Statistics, 32(4), 599–616.

    Article  MathSciNet  MATH  Google Scholar 

  11. Cao, R., & Van Keilegom, I. (2006). Empirical likelihood tests for two-sample problems via nonparametric density estimation. The Canadian Journal of Statistics, 1(34), 61–77.

    Article  MathSciNet  MATH  Google Scholar 

  12. Chen, H., & Friedman, J. H. (2017). A new graph-based two-sample test for multivariate and object data. Journal of the American statistical association, 112(517), 397–409.

    Article  MathSciNet  Google Scholar 

  13. Chen, L., Dou, W. W., & Qiao, Z. (2013). Ensemble subsampling for imbalanced multivariate two-sample tests. Journal of the American Statistical Association, 108(504), 1308–1323.

    Article  MathSciNet  MATH  Google Scholar 

  14. D’Agostino, R. B., & Stephen, M. A. (1986). Goodness-of-fit techniques. New York: Marcel Dekker.

    MATH  Google Scholar 

  15. Danafar, S., Rancoita, P., Glasmachers, T., Whittingstall, K., & Schmidhuber, J. (2014). Testing hypotheses by regularized maximum mean discrepancy. International Journal of Computer and Information Technology, 3(2), 223–232.

    Google Scholar 

  16. Durbin, J. (1973). Distribution theory for tests based on the sample distribution function. CBMS-NSF, Regional Conference Series in Applied Mathematics. Philadelphia: SIAM.

    Google Scholar 

  17. Eric, M., Bach, F. R., & Harchaoui, Z. (2008). Testing for homogeneity with kernel fisher discriminant analysis. In Advances in neural information processing systems (Vol. 20, pp. 609–616).

    Google Scholar 

  18. Fernández, V. A., Gamero, M. J., & García, J. M. (2008). A test for the two-sample problem based on empirical characteristic functions. Computational Statistics & Data Analysis, 52(7), 3730–3748.

    Article  MathSciNet  MATH  Google Scholar 

  19. Friedman, J. H., & Rafsky, L. C. (1979). Multivariate generalizations of the Wald-Wolfowitz and Smirnov two-sample tests. The Annals of Statistics, 7(4), 697–717.

    Article  MathSciNet  MATH  Google Scholar 

  20. Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B., & Smola, A. (2012a). A kernel two-sample test. The Journal of Machine Learning Research, 13(1), 723–773.

    MathSciNet  MATH  Google Scholar 

  21. Gretton, A., Sejdinovic, D., Strathmann, H., Balakrishnan, S., Pontil, M., Fukumizu, K., & Sriperumbudur, B. K. (2012b). Optimal kernel choice for large-scale two-sample tests. Advances in Neural Information Processing Systems, 25, 1214–1222.

    Google Scholar 

  22. Hall, P., & Tajvidi, N. (2002). Permutation tests for equality of distributions in high dimensional settings. Biometrika, 89(2), 359–374.

    Article  MathSciNet  MATH  Google Scholar 

  23. Henze, N. (1984). On the number of random points with nearest neighbors of the same type and a multivariate two-sample test. Metrika, 31, 259–273.

    Article  MathSciNet  Google Scholar 

  24. Henze, N. (1988). A multivariate two-sample test based on the number of nearest neighbor type coincidences. The Annals of Statistics, 16(2), 772–783.

    Article  MathSciNet  MATH  Google Scholar 

  25. Huo, X., & Székely, G. J. (2016). Fast computing for distance covariance. Technometrics, 58(4), 435–447.

    Article  MathSciNet  Google Scholar 

  26. Karoui, N. E., & Purdom, E. (2016). Can we trust the bootstrap in high-dimension? arXiv:1608.00696.

    Google Scholar 

  27. Lindsay, B. G., Markatou, M., & Ray, S. (2014). Kernels, degrees of freedom, and power properties of quadratic distance goodness-of-fit tests. Journal of the American Statistical Association, 109(505), 395–410.

    Article  MathSciNet  MATH  Google Scholar 

  28. Lindsay, B. G., Markatou, M., Ray, S., Yang, K., & Chen, S. C. (2008). Quadratic distances on probabilities: A unified foundation. The Annals of Statistics, 36(2), 983–1006.

    Article  MathSciNet  MATH  Google Scholar 

  29. Liu, Z., & Modarres, R. (2011). A triangle test for equality of distribution functions in high dimensions. Journal of Nonparametric Statistics, 23(3), 605–615.

    Article  MathSciNet  MATH  Google Scholar 

  30. Lyons, R. (2013). Distance covariance in metrics spaces. The Annals of Statistics, 41(5), 3284–3305.

    MathSciNet  MATH  Google Scholar 

  31. Maag, U. R., & Stephens, M. A. (1968). The v nm two-sample test. The Annals of Mathematical Statistics, 39(3), 923–935.

    Article  MathSciNet  MATH  Google Scholar 

  32. Mondal, P. K., Biswas, M., & Ghosh, A. K. (2015). On high dimensional two-sample tests based on nearest neighbors. Journal of Multivariate Analysis, 141, 168–178.

    Article  MathSciNet  MATH  Google Scholar 

  33. Muandet, K., Fukumizu, K., Sriperumbudur, B., & Schölkopf, B. (2017). Kernel mean embedding of distributions: A review and beyond. Foundations and Trends in Machine Learning, 10(1), 1–141.

    Article  MATH  Google Scholar 

  34. Neuhaus, G. (1977). Functional limit theorems for u-statistics in the degenerate case. Journal of Multivariate Analysis, 7(3), 424–439.

    Article  MathSciNet  MATH  Google Scholar 

  35. Pettitt, A. N. (1976a). A two-sample anderson-darling rank statistic. Biometrika, 63(1), 161–168.

    MathSciNet  MATH  Google Scholar 

  36. Pettitt, A. N. (1976b). Two-sample cramér-von mises type rank statistics. Journal of the Royal Statistical Society: Series B, 41(1), 46–53.

    MATH  Google Scholar 

  37. Póczos, B., Ghahramani, Z., & Schneider, J. (2012). Copula-based kernel dependency measures. Proceedings of the 29th International Conference on Machine Learning (pp. 775–782).

    Google Scholar 

  38. Popper, K. R. (1986). The logic of scientific discovery (2nd ed.). New York: Harper and Row.

    MATH  Google Scholar 

  39. Rayner, J. C. W., & Best, D. J. (1986). Smooth tests of goodness of fit. Oxford: Oxford University Press.

    MATH  Google Scholar 

  40. Reddi, S., Ramdas, A., Póczos, B., Singh, A., & Wasserman, L. (2015). On the decreasing power of kernel and distance based nonparametric hypothesis tests in high dimensions. Proceedings of 29th AAAI Conference on Artificial Intelligence (pp. 3571–3577).

    Google Scholar 

  41. Rosenbaum, P. R. (2005). An exact distribution free test comparing two multivariate distributions based on adjacency. Journal of the Royal Statistical Society: Series B, 67(4), 515–530.

    Article  MathSciNet  MATH  Google Scholar 

  42. Savage, L. T. (1972). The foundations of statistics (2nd ed.). New York: Dover.

    MATH  Google Scholar 

  43. Schilling, M. F. (1986). Multivariate two-sample tests based on nearest neighbors. Journal of the American Statistical Association, 81(395), 799–806.

    Article  MathSciNet  MATH  Google Scholar 

  44. Sejdinovic, D., Surperumbudur, B., Gretton, A., & Fukumizu, K. (2013). Equivalence of distance-based and RKHS-based statistics in hypothesis testing. The Annals of Statistics, 41(3), 2263–2291.

    Article  MathSciNet  MATH  Google Scholar 

  45. Szabo, A., Boucher, K., Carroll, W. L., Klebanov, L. B., Tsodikov, A. D., & Yakovlev, A. Y. (2002). Variable selection and pattern recognition with gene expression data generated by the microarray technology. Mathematical Biosciences, 176(1), 71–98.

    Article  MathSciNet  MATH  Google Scholar 

  46. Szabo, A., Boucher, K., Jones, D., Tsodikov, A. D., Klebanov, L. B., & Yakovlev, A. Y. (2003). Multivariate exploratory tools for microarray data analysis. Biostatistics, 4(4), 555–567.

    Article  MATH  Google Scholar 

  47. Székely, G. J., & Rizzo, M. L. (2004). Testing for equal distributions in high dimension. InterStat, 5(1), 1–6.

    Google Scholar 

  48. Székely, G. J., & Rizzo, M. L. (2005). A new test for multivariate normality. Journal of Multivariate Analysis, 93(1), 58–80.

    Article  MathSciNet  MATH  Google Scholar 

  49. Székely, G. J., & Rizzo, M. L. (2013). Energy statistics: A class of statistics based on distances. Journal of Statistical Planning and Inference, 143(8), 1249–1272.

    Article  MathSciNet  MATH  Google Scholar 

  50. Wald, A. (1950). Statistical decision functions. New York: John Wiley & Sons.

    MATH  Google Scholar 

  51. Wald, A., & Wolfowitz, J. (1940). On a test whether two samples are from the same population. The Annals of Mathematical Statistics, 11(2), 147–162.

    Article  MathSciNet  MATH  Google Scholar 

  52. Wu, M. C., Lee, S., Cai, T., Li, Y., Boehnke, M., & Lin, X. (2011). Rare-variant association testing for sequencing data with the sequence kernel association test. The American Journal of Human Genetics, 89(1), 82–93.

    Article  Google Scholar 

  53. Zaremba, W., Gretton, A., & Blaschko, M. (2013). B-test: A non-parametric, low variance kernel two-sample test. Advances in neural information processing systems (pp. 755–763)

    Google Scholar 

Download references

Acknowledgement

The work of both authors is supported by The Troup Fund of the Kaleida Health Foundation.

Author information

Authors and Affiliations

  1. Department of Biostatistics, University at Buffalo, Buffalo, NY, USA

    Yang Chen & Marianthi Markatou

Authors
  1. Yang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marianthi Markatou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marianthi Markatou .

Editor information

Editors and Affiliations

  1. Math and Statistics, 1342, Georgia State University, Atlanta, GA, USA

    Yichuan Zhao

  2. School of Social Work, University of North Carolina, Chapel Hill, NC, USA

    Ding-Geng (Din) Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Chen, Y., Markatou, M. (2020). Kernel Tests for One, Two, and K-Sample Goodness-of-Fit: State of the Art and Implementation Considerations. In: Zhao, Y., Chen, DG. (eds) Statistical Modeling in Biomedical Research. Emerging Topics in Statistics and Biostatistics . Springer, Cham. https://doi.org/10.1007/978-3-030-33416-1_14

Download citation

Publish with us

Policies and ethics

Search

Navigation