Nonparametric “anti-Bayesian” quantile-based pattern classification

Abstract

Parametric and nonparametric pattern recognition have been studied for almost a century based on a Bayesian paradigm, which is, in turn, founded on the principles of Bayes theorem. It is well known that the accuracy of the Bayes classifier cannot be exceeded. Typically, this reduces to comparing the testing sample to mean or median of the respective distributions. Recently, Oommen and his co-authors have presented a pioneering and non-intuitive paradigm, namely that of achieving the classification by comparing the testing sample with another descriptor, which could also be quite distant from the mean. This paradigm has been termed as being “anti-Bayesian,” and it essentially uses the quantiles of the distributions to achieve the pattern recognition. Such classifiers attain the optimal Bayesian accuracy for symmetric distributions even though they operate with a non-intuitive philosophy. While this paradigm has been applied in a number of domains (briefly explained in the body of this paper), its application for nonparametric domains has been limited. This paper explains, in detail, how such quantile-based classification can be extended to the nonparametric world, using both traditional and kernel-based strategies. The paper analyzes the methodology of such nonparametric schemes and their robustness. From a fundamental perspective, the paper utilizes the so-called large sample theory to derive strong asymptotic results that pertain to the equivalence between the parametric and nonparametric schemes for large samples. Apart from the new theoretical results, the paper also presents experimental results demonstrating their power. These results pertain to artificial data sets and also involve a real-life breast cancer data set obtained from the University Hospital Centre of Coimbra. The experimental results clearly confirm the power of the proposed “anti-Bayesian” procedure, especially when approached from a nonparametric perspective.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

Notes

  1. 1.

    In the last century, there are, indeed, tens of thousands of papers describing the art and science of Bayesian classification—for a myriad of distributions and applications. In this paper, we do not attempt a survey of the field.

  2. 2.

    We are very grateful to the anonymous referee of the previous version of the paper, who requested this.

  3. 3.

    Initially, the authors of [31] stated that the classification was based on the order statistics of the distribution, and this was later rectified [33].

  4. 4.

    With going into too many details, we refer the reader to [5], which is a key reference in this field.

  5. 5.

    To be fair to the authors of [12, 22, 34, 35], one must grant them the credit that they were able to achieve their nonparametric results by using the “anti-Bayesian” paradigm in multidimensions, as opposed to unidimensions, as we have done here!

  6. 6.

    The proof of the theorem is omitted, since it is found in the literature. Also, we refer the interested reader to [6] for more information about the various types of convergence.

  7. 7.

    It is pertinent to mention that the accuracy of any classifier can and will never exceed that of a Bayesian classifier. The amazing thing is that we have been able to attain to an accuracy quite close to the optimal, even though we have worked in a counterintuitive manner, and also made no assumption about the underlying distribution!

  8. 8.

    For more details about outliers in statistical analysis, we refer the reader to [8, 16, 25, 27].

  9. 9.

    The data may be obtained from the UCI Repository of Machine Learning databases at archive.ics.uci.edu/ml.

References

  1. 1.

    Ahsanullah M, Nevzorov VB (2005) Order statistics: examples and exercises. Nova Publishers, Hauppauge

    Google Scholar 

  2. 2.

    Aitkin M, Wilson GT (1980) Mixture models, outliers, and the EM algorithm. Technometrics 22(3):325–331

    MATH  Google Scholar 

  3. 3.

    Alfred M (2001) Stochastic ordering of multivariate normal distributions. Ann Inst Stat Math 53(3):567–575

    MathSciNet  MATH  Google Scholar 

  4. 4.

    Altman N, Léger C (1995) Bandwidth selection for kernel distribution function estimation. J Stat Plan Inference 46(2):195–214

    MathSciNet  MATH  Google Scholar 

  5. 5.

    Arnold BC, Balakrishnan N, Nagaraja HN (2008) A first course in order statistics. SIAM, Philadelphia

    Google Scholar 

  6. 6.

    Athreya KB, Lahiri SN (2006) Measure theory and probability theory. Springer, Berlin

    Google Scholar 

  7. 7.

    Azzalini A (1981) A note on the estimation of a distribution function and quantiles by a kernel method. Biometrika 68(1):326–328

    MathSciNet  Google Scholar 

  8. 8.

    Barnett V, Lewis T (1978) Outliers in statistical data. Wiley, Hoboken

    Google Scholar 

  9. 9.

    Binder DA (1978) Bayesian cluster analysis. Biometrika 65(1):31–38

    MathSciNet  MATH  Google Scholar 

  10. 10.

    Bishop CM (2006) Pattern recognition and machine learning (information science and statistics). Springer, Berlin, Heidelberg

    Google Scholar 

  11. 11.

    David HA, Nagaraja HN (2004) Order statistics. Wiley, Hoboken

    Google Scholar 

  12. 12.

    Hammer H, Yazidi A, Oommen BJ (2017) “Anti-Bayesian” flat and hierarchical clustering using symmetric quantiloids. Inf Sci 418–419:495–512

    MathSciNet  Google Scholar 

  13. 13.

    Hawkins DM (1980) Identification of outliers. Chapman and Hall, London

    Google Scholar 

  14. 14.

    Hollander M, Wolfe DA, Chicken E (2013) Nonparametric statistical methods. Wiley, Hoboken

    Google Scholar 

  15. 15.

    Hu L (2015) A note on order statistics-based parametric pattern classification. Pattern Recognit 48(1):43–49

    MATH  Google Scholar 

  16. 16.

    Huber PJ (2011) Robust statistics. In: International encyclopedia of statistical science. pp 1248–1251

  17. 17.

    Kothari CR (2004) Research methodology: methods and techniques. New Age International, New Delhi

    Google Scholar 

  18. 18.

    Leech NL, Onwuegbuzie AJ (2002) A call for greater use of nonparametric statistics. In: Mid-south educational research association annual meeting

  19. 19.

    Meegen CV, Schnackenberg S, Ligges U (2019) Unequal priors in linear discriminant analysis. J Classif. https://doi.org/10.1007/s00357-019-09336-2

    Google Scholar 

  20. 20.

    Nguyen-Trang T, Vo-Van T (2017) A new approach for determining the prior probabilities in the classification problem by Bayesian method. Adv Data Anal Classif 11:629–643

    MathSciNet  MATH  Google Scholar 

  21. 21.

    Oommen BJ, Thomas A (2014) Optimal order statistics-based “anti-Bayesian” parametric pattern classification for the exponential family. Pattern Recognit 47:40–55

    MATH  Google Scholar 

  22. 22.

    Oommen BJ, Khoury R, Schmidt A (2015) Text classification using novel “anti-Bayesian” techniques. In: Nunez M, Nguyen N, Camacho D, Trawinski B (eds) Computational collective intelligence. Lecture notes in computer science, vol 9329. pp 1–15

  23. 23.

    Patrício M, Pereira J, Crisóstomo J, Matafome P, Gomes M, Seiça R, Caramelo F (2018) Using resistin, glucose, age and BMI to predict the presence of breast cancer. BMC Cancer 18:29

    Google Scholar 

  24. 24.

    Rosenblatt M (1956) Remarks on some nonparametric estimates of a density function. Ann Math Stat 27(3):832–837

    MathSciNet  MATH  Google Scholar 

  25. 25.

    Rousseeuw PJ, Leroy AM (2005) Robust regression and outlier detection. Wiley, Hoboken

    Google Scholar 

  26. 26.

    Santhanam V, Morariu VI, Harwood D, Davis LS (2016) A non-parametric approach to extending generic binary classifiers for multi-classification. Pattern Recognit 58:149–158

    Google Scholar 

  27. 27.

    Scott D W (2004) Partial mixture estimation and outlier detection in data and regression. In: Hubert M, Pison G, Struyf A, Van Aelst S (eds) Theory and applications of recent robust methods. Statistics for industry and technology. pp 297–306

  28. 28.

    Serfling RJ (2009) Approximation theorems of mathematical statistics. Wiley, Hoboken

    Google Scholar 

  29. 29.

    Shaked M, Shanthikumar JG (2007) Stochastic orders. Springer, Berlin

    Google Scholar 

  30. 30.

    Thomas A, Oommen BJ (2012) Optimal “anti-Bayesian” parametric pattern classification for the exponential family using order statistics criteria. In: Alvarez L, Mejail M, Gomez L, Jacobo J (eds) Progress in pattern recognition, image analysis, computer vision, and applications. CIARP 2012. Lecture notes in computer science, vol 7441. pp 1–13

  31. 31.

    Thomas A, Oommen BJ (2013) The fundamental theory of optimal “anti-Bayesian” parametric pattern classification using order statistics criteria. Pattern Recognit 46(1):376–388

    MATH  Google Scholar 

  32. 32.

    Thomas A, Oommen B J (2013) Order statistics-based parametric classification for multi-dimensional distributions. Pattern Recognit 46(12):3472–3482

    MATH  Google Scholar 

  33. 33.

    Thomas A, Oommen BJ (2014) Corrigendum to three papers that deal with “anti-Bayesian” pattern recognition. Pattern Recognit 47(6):2301–2302

    MATH  Google Scholar 

  34. 34.

    Thomas A, Oommen BJ (2013) Ultimate order statistics-based prototype reduction schemes. In: Cranefield S, Nayak A (eds) AI 2013: Advances in artificial intelligence. AI 2013. Lecture notes in computer science, vol 8272. pp 421–433

  35. 35.

    Thomas A, Oommen BJ (2013) A novel Border Identification algorithm based on an “anti-Bayesian” paradigm. In: Wilson R, Hancock E, Bors A, Smith W (eds) Computer analysis of images and patterns. CAIP 2013. Lecture notes in computer science, vol 8047. pp 196–203

Download references

Acknowledgements

We are very grateful to the anonymous referees of the previous version of the paper, who suggested various modifications and changes. Their suggestions have greatly enhanced the quality of this present version.

Author information

Affiliations

Authors

Corresponding author

Correspondence to B. John Oommen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mahmoudi, F., Razmkhah, M. & Oommen, B.J. Nonparametric “anti-Bayesian” quantile-based pattern classification. Pattern Anal Applic (2020). https://doi.org/10.1007/s10044-020-00903-7

Download citation

Keywords

  • “Anti-Bayesian” classification
  • Nonparametric quantile-based method
  • Mixture model
  • Sample quantile
  • Kernel density estimation
  • Robust classification