Advertisement

Statistical Papers

, Volume 60, Issue 3, pp 903–931 | Cite as

Classification with the pot–pot plot

  • Oleksii PokotyloEmail author
  • Karl Mosler
Regular Article

Abstract

We propose a procedure for supervised classification that is based on potential functions. The potential of a class is defined as a kernel density estimate multiplied by the class’s prior probability. The method transforms the data to a potential–potential (pot–pot) plot, where each data point is mapped to a vector of potentials. Separation of the classes, as well as classification of new data points, is performed on this plot. For this, either the \(\alpha \)-procedure (\(\alpha \)-P) or k-nearest neighbors (k-NN) are employed. For data that are generated from continuous distributions, these classifiers prove to be strongly Bayes-consistent. The potentials depend on the kernel and its bandwidth used in the density estimate. We investigate several variants of bandwidth selection, including joint and separate pre-scaling and a bandwidth regression approach. The new method is applied to benchmark data from the literature, including simulated data sets as well as 50 sets of real data. It compares favorably to known classification methods such as LDA, QDA, max kernel density estimates, k-NN, and DD-plot classification using depth functions.

Keywords

Kernel density estimates Bandwidth choice Potential functions k-Nearest-neighbors classification \(\alpha \)-Procedure DD-plot \(DD\alpha \)-classifier 

Mathematics Subject Classification

62H30 62G07 

Notes

Acknowledgements

We are grateful to Tatjana Lange and Pavlo Mozharovskyi for the active discussion of this paper. The work of Oleksii Pokotylo is supported by the Cologne Graduate School of Management, Economics and Social Sciences.

Supplementary material

362_2016_854_MOESM1_ESM.pdf (253 kb)
Supplementary material 1 (pdf 253 KB)

References

  1. Aizerman MA, Braverman EM, Rozonoer LI (1970) The method of potential functions in the theory of machine learning. Nauka, MoscowzbMATHGoogle Scholar
  2. Chang C-C, Lin C-J (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2: 27:1–27:27. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm
  3. Cuesta-Albertos JA, Febrero-Bande M, de la Fuente MO (2016) The DD\(^G\)-classifier in the functional setting. arXiv:1501.00372
  4. Cuevas A, Febrero M, Fraiman R (2007) Robust estimation and classification for functional data via projection-based depth notions. Comput Stat 22(3):481–496MathSciNetCrossRefzbMATHGoogle Scholar
  5. Devroye L, Györfi L, Lugosi G (1996) A probabilistic theory of pattern recognition. Springer, New YorkCrossRefzbMATHGoogle Scholar
  6. Duong T (2007) ks: kernel density estimation and kernel discriminant analysis for multivariate data in R. J Stat Softw 21:1–16CrossRefGoogle Scholar
  7. Dutta S, Chaudhuri P, Ghosh AK (2012) Classification using localized spatial depth with multiple localization. Mimeo, New YorkGoogle Scholar
  8. Fraiman R, Meloche J (1999) Multivariate L-estimation. Test 8:255–317MathSciNetCrossRefzbMATHGoogle Scholar
  9. Friedman JH (1997) On bias, variance, 0/1-loss, and the curse-of-dimensionality. Data Min Knowl Discov 1:55–77CrossRefGoogle Scholar
  10. Härdle W, Müller M, Sperlich S, Werwatz A (2004) Nonparametric and semiparametric models. Springer, New YorkCrossRefzbMATHGoogle Scholar
  11. Lange T, Mosler K, Mozharovskyi P (2014) Fast nonparametric classification based on data depth. Stat Pap 55:49–69MathSciNetCrossRefzbMATHGoogle Scholar
  12. Li J, Cuesta-Albertos JA, Liu RY (2012) DD-classifier: nonparametric classification procedure based on DD-plot. J Am Stat Assoc 107:737–753MathSciNetCrossRefzbMATHGoogle Scholar
  13. Mosler K (2013) Depth statistics. In: Becker C, Fried R, Kuhnt S (eds) Robustness and complex data structures: Festschrift in Honour of Ursula Gather. Springer, Berlin, pp 17–34CrossRefGoogle Scholar
  14. Mozharovskyi P, Mosler K, Lange T (2015) Classifying real-world data with the \(DD\alpha \)-procedure. Adv Data Anal Classif 9:287–314MathSciNetCrossRefzbMATHGoogle Scholar
  15. Paindaveine D, Van Bever G (2013) From depth to local depth: a focus on centrality. J Am Stat Assoc 108:1105–1119MathSciNetCrossRefzbMATHGoogle Scholar
  16. Paindaveine D, Van Bever G (2015) Nonparametrically consistent depth-based classifiers. Bernoulli 21:62–82MathSciNetCrossRefzbMATHGoogle Scholar
  17. Pokotylo O, Mozharovskyi P, Dyckerhoff R (2016) Depth and depth-based classification with R-package ddalpha. arXiv:1608.04109
  18. Scott DW (1992) Multivariate density estimation: theory, practice, and visualization. Wiley, New YorkCrossRefzbMATHGoogle Scholar
  19. Serfling R (2006) Depth functions in nonparametric multivariate inference. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol 72Google Scholar
  20. Silverman BW (1986) Density estimation for statistics and data analysis. Chapman and Hall, LondonCrossRefzbMATHGoogle Scholar
  21. Vencalek O (2014) New depth-based modification of the k-nearest neighbour method. SOP Trans Stat Anal 1:131–138CrossRefGoogle Scholar
  22. Wand MP, Jones MC (1993) Comparison of smoothing parameterizations in bivariate kernel density estimation. J Am Stat Assoc 88:520–528MathSciNetCrossRefzbMATHGoogle Scholar
  23. Zuo Y, Serfling R (2000) General notions of statistical depth function. Ann Stat 28:461–482MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Statistics and EconometricsUniversität zu KölnCologneGermany

Personalised recommendations