Norm ball classifier for one-class classification

Abstract

One-Class Classification (OCC) is a supervised learning technique for classification whereby the classifier is obtained only by training the objects from the target class and identifying whether new observations belong to the class or not. In this paper, we propose a novel approach to OCC, which is based on optimal covering of the target objects by ‘good’ norm balls. The proposed classifier consists of the selected norm balls from an integer programming model where the finite norm ball candidates from the target objects are used. Computational experiments were carried out to examine the performance and characteristics of the proposed classifier using artificial and real data from the UCI Repository. The results showed that the proposed model was comparable to existing OCC methods in the comparison group. In addition, the proposed model demonstrated high sparsity leading to low testing burden and robustness to noises.

This is a preview of subscription content, access via your institution.

Fig.1
Fig. 2
Fig. 3
Fig. 4

References

  1. Beck, M. T., Werner, M., Feld, S., & Schimper, T. (2014). Mobile edge computing: A taxonomy

  2. Bishop, C. M. (1994). Novelty detection and neural network validation. IEE Proceedings-Vision, Image and Signal processing, 141, 217–222.

    Article  Google Scholar 

  3. Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford: Oxford University Press.

    Google Scholar 

  4. Cabral, G. G., Oliveira, A. L., & Cahu, C. B. (2009). Combining nearest neighbor data description and structural risk minimization for one-class classification. Neural Computing and Applications, 18, 175–183.

    Article  Google Scholar 

  5. Chen, H. S., & Becerra, V. M. (2016). Sparse density estimator with tunable kernels. Neurocomputing, 173, 1976–1982.

    Article  Google Scholar 

  6. Conforti, M., Cornuejols, G., & Zambelli, G. (2014). Integer programming (Vol. 271). Berlin: Springer.

    Google Scholar 

  7. Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern classification. New Jersy: John Wiley and Sons.

    Google Scholar 

  8. Eckstein, J., Hammer, P. L., Liu, Y., Nediak, M., & Simeone, B. (2002). The maximum box problem and its application to data analysis. Computational Optimization and Applications, 23, 285–298.

    Article  Google Scholar 

  9. Gaber, M. M., Gomes, J. B., & Stahl, F. (2014). Pocket data mining, big data on small devices. Series: Studies in big data

  10. Hastie, T., Tibshirani, R., & Wainwright, M. (2015). Statistical learning with sparsity. Boca Raton: CRC Press.

    Google Scholar 

  11. Japkowicz, N., Myers, C., Gluck, M., et al. (1995). A novelty detection approach to classification. IJCAI, 1, 518–523.

    Google Scholar 

  12. Jeong, K., & Choi, J. Y. (2015). Design of one-class classifier using hyper-rectangles. Journal of Korean Institute of Industrial Engineers, 41, 439–446.

    Article  Google Scholar 

  13. Letouzey, F., Denis, F., & Gilleron, R. (2000). Learning from positive and unlabeled examples, in Algorithmic Learning Theory (pp. 71–85). Berlin: Springer.

    Google Scholar 

  14. Li, C., Zhang, Y., & Li, X. (2009). Ocvfdt: one-class very fast decision tree for one-class classification of data streams, In Proceedings of the Third International Workshop on Knowledge Discovery from Sensor Data, ACM, pp. 79–86.

  15. Lichman, M. (2013). UCI machine learning repository, http://archive.ics.uci.edu/ml

  16. Manevitz, L., & Yousef, M. (2007). One-class document classification via neural networks. Neurocomputing, 70, 1466–1481.

    Article  Google Scholar 

  17. Nguyen, G.H., Bouzerdoum, A., & Phung, S.L. (2009). Learning pattern classification tasks with imbalanced data sets, in Pattern Recognition, InTCH

  18. Parzen, E. (1962). On estimation of a probability density function and mode. The annals of mathematical statistics, 33, 1065–1076.

    Article  Google Scholar 

  19. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.

    Google Scholar 

  20. Ritter, G., & Gallegos, M. T. (1997). Outliers in statistical pattern recognition and an application to automatic chromosome classification. Pattern Recognition Letters, 18, 525–539.

    Article  Google Scholar 

  21. Schölkopf, B., Williamson, R. C., Smola, A. J., Shawe-Taylor, J., Platt, J. C., et al. (1999). Support vector method for novelty detection. NIPS, 12, 582–588.

    Google Scholar 

  22. Serafini, P. (2014). Classifying negative and positive points by optimal box clustering. Discrete Applied Mathematics, 165, 270–282.

    Article  Google Scholar 

  23. Skabar, A. (2003). Single-class classifier learning using neural networks: An application to the prediction of mineral deposits. In Machine Learning and Cybernetics, 2003 International Conference on, IEEE, Vol. 4, pp. 2127–2132.

  24. Tax, D.M. (2001) One-class classification, Ph.D Thesis, Delft University of Technology

  25. Tax, D. M., & Duin, R. P. (1999). Data domain description using suppor vectors. ESANN, 99, 251–256.

    Google Scholar 

  26. Tax, D. M., & Duin, R. P. (2000). Data description in subspaces, in Pattern Recognition, 2000. Proceedings. 15th International Conference on, vol. 2, IEEE, pp. 672–675.

  27. Tax, D. M., & Duin, R. P. (2004). Support vector data description. Machine learning, 54, 45–66.

    Article  Google Scholar 

  28. Thornhill, N. F., Patwardhan, S. C., & Shah, S. L. (2008). A continuous stirred tank heater simulation model with applications. Journal of Process Control, 18, 347–360.

    Article  Google Scholar 

  29. Thornhill, N. F., Patwardhan, S. C., & Shah, S. L. (2013). The continuous stirred tank heater simulation, http://personalpages.ps.ic.ac.uk/~nina/CSTHSimulation/index.htm.

  30. Tsai, C. F., Hsu, Y. F., Lin, C. Y., & Lin, W. Y. (2009). Intrusion detection by machine learning: A review. Expert Systems with Applications, 36, 11994–12000.

    Article  Google Scholar 

  31. Xpress, (2017). Xpress 8.0, http://www.fico.com/en.

  32. Yu, H., Han, J., & Chang, K.-C. (2004). Pebl: Web page classification without negative examples. IEEE Transactions on Knowledge and Data Engineering, 16, 70–81.

    Article  Google Scholar 

  33. Yu, H., Zhai, C., & Han, J. (2003). Text classification from positive and unlabeled documents, In Proceedings of the twelfth international conference on Information and knowledge management, ACM, pp. 232–239.

  34. Zhou, J., Chan, K., Chong, V., & Krishnan, S. M. (2006). Extraction of brain tumor from mri images using one-class support vector machine, in Engineering in Medicine and Biology Society, 2005. IEEE-EMBS 2005. 27th Annual International Conference of the, IEEE, pp. 6411-6414

Download references

Acknowledgements

The authors are grateful for the valuable comments from anonymous reviewers. This work was supported in part by the National Research Foundation of Korea Grant (No. NRF-2019R1F1A1042307) and in part by the National Research Foundation of Korea Grant (No. NRF-2018R1A2B2003227). In addition, this work was supported in part by BK21 FOUR (Brain Korea 21 Fostering Outstanding Universities for Research) in Interdisciplinary Program of Arts & Design Technology, Chonnam National University.

Author information

Affiliations

Authors

Corresponding authors

Correspondence to Kyungsik Lee or Young-Seon Jeong.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Tables 8, 9 , 10, 11, 12, 13, 14, 15, 16, 17, 18 .

Table 9 The average recall, precision and \(F_{1}\)-score of Bimodal data depending on the noise level (standard deviation in parentheses)
Table 10 The average recall, precision and \(F_{1}\)-score of CSTH data depending on the noise level (standard deviation in parentheses)
Table 11 The average recall, precision and \(F_{1}\)-score of Iris-setosa data depending on the noise level (standard deviation in parentheses)
Table 12 The average recall, precision and \(F_{1}\)-score of Iris-versicolor data depending on the noise level (standard deviation in parentheses)
Table 13 The average recall, precision and \(F_{1}\) \({{\varvec{F}}}_{1}\)-score of Iris-virginica data depending on the noise level (standard deviation in parentheses)
Table 14 The average recall, precision and \(F_{1}\)-score of Liver data depending on the noise level (standard deviation in parentheses)
Table 15 The average recall, precision and \(F_{1}\)-score of WBC data depending on the noise level (standard deviation in parentheses)
Table 16 The average recall, precision and \(F_{1}\)-score of Ionosphere data depending on the noise level (standard deviation in parentheses)
Table 17 Sparsity of classifiers in the test for Bimodal, CSTH, Iris-setosa and versicolor data sets on the noise level (standard deviation in parentheses)
Table 18 Sparsity of classifiers in the test for Iris-virginica, Liver, WBC and Ionosphere data sets on the noise level (standard deviation in parentheses)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kim, S., Lee, K. & Jeong, YS. Norm ball classifier for one-class classification. Ann Oper Res (2021). https://doi.org/10.1007/s10479-021-03964-x

Download citation

Keywords

  • Norm ball
  • One-class classification
  • Integer programming
  • Set covering problem