Symmetrizing k-nn and Mutual k-nn Smoothers

  • P.-A. Cornillon
  • A. Gribinski
  • N. Hengartner
  • T. Kerdreux
  • E. Matzner-LøberEmail author
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 250)


In light of Cohen (Ann Math Stat 37:458–463, 1966) and Rao (Ann Stat 4:1023–1037, 1976), who provide necessary and sufficient conditions for admissibility of linear smoothers, one realizes that many of the well-known linear nonparametric regression smoothers are inadmissible because either the smoothing matrix is asymmetric or the spectrum of the smoothing matrix lies outside the unit interval [0, 1]. The question answered in this chapter is how can an inadmissible smoother transformed into an admissible one? Specifically, this contribution investigates the spectrum of various matrix symmetrization schemes for k-nearest neighbor-type smoothers. This is not an easy task, as the spectrum of many traditional symmetrization schemes fails to lie in the unit interval. The contribution of this study is to present a symmetrization scheme for smoothing matrices that make the associated estimator admissible. For k-nearest neighbor smoothers, the result of the transformation has a natural interpretation in terms of graph theory.


  1. 1.
    Biau, G., & Devroye, L. (2015). Lectures on the nearest neighbor method. Cham: Springer.Google Scholar
  2. 2.
    Brito, M. R., Chavez, E. L., Quiroz, A. J., & Yukisk, J. E. (1997). Connectivity of the mutual k-nearest-neighbor graph in clustering and outlier detection. Statistics and Probability Letters, 35(1), 33–42.Google Scholar
  3. 3.
    Chan, S., Zickler, T., & Lu, Y. (2015). Understanding symmetric smoothing filters via Gaussian mixtures. In 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC (pp. 2500–2504).Google Scholar
  4. 4.
    Chan, S., Zickler, T., & Lu, Y. (2017). Understanding symmetric smoothing filters via Gaussian mixtures. IEEE Transactions on Image Processing, 26(11), 5107–5121.Google Scholar
  5. 5.
    Cohen, A. (1966). All admissible linear estimates of the mean vector. Annals of Mathematical Statistics, 37, 458–463.Google Scholar
  6. 6.
    Cornillon, P. A., Hengartner, N., & Matzner-Løber, E. (2013). Recursive bias estimation for multivariate regression smoothers. ESAIM: Probability and Statistics, 18, 483–502.Google Scholar
  7. 7.
    Devroye, L., Györfi, L., & Lugosi, G. (1996). A probabilistic theory of pattern recognition. New York: Springer.Google Scholar
  8. 8.
    Dong, W., Moses, C., & Li, K. (2011). Efficient k-nearest neighbor graph construction for generic similarity measures. In Proceedings of the 20th International Conference on World Wide Web (pp. 577–586).Google Scholar
  9. 9.
    Eubank, R. (1999). Nonparametric regression and spline smoothing (2nd ed.). New York: Dekker.Google Scholar
  10. 10.
    Fan, J., & Gijbels, I. (1996). Local polynomial modeling and its application, theory and methodologies. New York, NY: Chapman et Hall.Google Scholar
  11. 11.
    Fix, E., & Hodges, J. L. (1951). Discriminatory Analysis, Nonparametric Discrimination: Consistency Properties. Technical report, USAF School of Aviation Medicine, Randolph Field, TX.Google Scholar
  12. 12.
    Friedman, J., Hastie, T., & Tibshirani, R. (2000). Additive logistic regression: A statistical view of boosting. The Annals of Statistics, 28, 337–407.Google Scholar
  13. 13.
    Gowda, K., & Krishna, G. (1978). Agglomerative clustering using the concept of mutual nearest neighbourhood. Pattern Recognition, 10 (2), 105–112.Google Scholar
  14. 14.
    Guyader, A., & Hengartner, N. (2013). On the mutual nearest neighbors estimate in regression. The Journal of Machine Learning Research (JMLR), 13, 2287–2302.Google Scholar
  15. 15.
    Gyorfi, L., Kohler, M., Krzyzak, A., & Walk, H. (2002). A distribution-free theory of nonparametric regression. New York: Springer.Google Scholar
  16. 16.
    Haque, S., Pai, G., & Govindu, V. (2014). Symmetric smoothing filters from global consistency constraints. IEEE Transactions on Image Processing, 24, 1536–1548.Google Scholar
  17. 17.
    Kessler, K. (1963). Bibliographic coupling between scientific papers. American Documentation, 14, 10–25.Google Scholar
  18. 18.
    Knight, P. (2008). The Sinkhorn-Knopp algorithm: Convergence and applications. SIAM Journal on Matrix Analysis and Applications, 30, 261–275.Google Scholar
  19. 19.
    Linton, O., & Jacho-Chavez, D. (2010). On internal corrected and symmetrized kernel estimators for nonparametric regression. Test, 19, 166–186.Google Scholar
  20. 20.
    Lovasz, L. (2010). Large networks and graph limits. London: CRC.Google Scholar
  21. 21.
    Maier, M., von Luxburg, U., & Hein, M. (2013). How the result of graph clustering methods depends on the construction of the graph. ESAIM Probability and Statistics, 17, 370–418.Google Scholar
  22. 22.
    Milanfar, P. (2013). Symmetrizing smoothing filters. SIAM Journal of Imaging Sciences, 6(1), 263–284.Google Scholar
  23. 23.
    Nadaraya, E. A. (1964). On estimating regression. Theory Probability and Their Applications, 9, 134–137.Google Scholar
  24. 24.
    Radovanović, M., Nanopoulos, A., & Ivanović, M. (2010). Hubs in space: Popular nearest neighbors in high-dimensional data. Journal of Machine Learning Research, 11, 2487–2531.Google Scholar
  25. 25.
    Rao, C. R. (1976). Estimation of parameters in linear models. The Annals of Statistics, 4, 1023–1037.Google Scholar
  26. 26.
    Sinkhorn, R. (1964). A relationship between arbitrary positive matrices and doubly stochastic matrices. The Annals of Mathematical Statistics, 35, 876–879.Google Scholar
  27. 27.
    Small, H. (1973). Co-citation in the scientific literature: A new measure of relationship between documents. Journal of the American Society for Information Science, 24, 265–269.Google Scholar
  28. 28.
    Zhao, L. (1999). Improved estimators in nonparametric regression problems. Journal of the American Statistical Association, 94 (445), 164–173.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • P.-A. Cornillon
    • 1
  • A. Gribinski
    • 2
  • N. Hengartner
    • 3
  • T. Kerdreux
    • 4
  • E. Matzner-Løber
    • 5
    Email author
  1. 1.University of RennesIRMAR UMR 6625RennesFrance
  2. 2.Department of MathematicsPrinceton UniversityPrincetonUSA
  3. 3.Los Alamos National LaboratoryLos AlamosUSA
  4. 4.UMR 8548Ecole Normale SupérieureParisFrance
  5. 5.CREST, UMR 9194, Cepe-EnsaePalaiseauFrance

Personalised recommendations