How to Evaluate Three-Way Decisions Based Binary Classification?

  • Xiuyi JiaEmail author
  • Lin Shang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9437)


Appropriate measures are important for evaluating the performance of a classifier. In existing studies, many performance measures designed for two-way decisions based classification are applied to three-way decisions based classification directly, which may result in an incomprehensive evaluation. However, there is a lack of systematically research on the performance measures for three-way decisions based classification. This paper introduces some numerical measures and graphical measures for three-way decisions based binary classification.


Three-way decisions Binary classification Performance measure 



We would like to acknowledge the support for this work from the National Natural Science Foundation of China (Grant Nos. 61403200, 61170180), Natural Science Foundation of Jiangsu Province(Grant No.BK20140800).


  1. 1.
    Fawcett, T.: An introduction to ROC analysis. Pattern Recogn. Lett. 27, 861–874 (2006)CrossRefGoogle Scholar
  2. 2.
    Gu, Q., Zhu, L., Cai, Z.H.: Evaluation measures of the classification performance of imbalanced data sets. In: Proceedings of the 4th International Symposium on Computational Intelligence and Intelligent Systems, pp. 461–471 (2009)Google Scholar
  3. 3.
    Jia, X.Y., Liao, W.H., Tang, Z.M., Shang, L.: Minimum cost attribute reduction in decision-theoretic rough set models. Inf. Sci. 219, 151–167 (2013)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Jia, X., Shang, L.: Three-Way Decisions Versus Two-Way Decisions on Filtering Spam Email. In: Peters, J.F., Skowron, A., Li, T., Yang, Y., Yao, J.T., Nguyen, H.S. (eds.) Transactions on Rough Sets XVIII. LNCS, vol. 8449, pp. 69–91. Springer, Heidelberg (2014) Google Scholar
  5. 5.
    Jia, X.Y., Shang, L., Zhou, B., Yao, Y.Y.: Generalized attribute reduct in rough set theory. Knowl.-Based Syst. 80, 67–77 (2015). doi: 10.1016/j.knosys.2015.05.017CrossRefGoogle Scholar
  6. 6.
    Jia, X.Y., Tang, Z.M., Liao, W.H., Shang, L.: On an optimization representation of decision-theoretic rough set model. Int. J. Approximate Reasoning 55(1), 156–166 (2014)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Li, W., Miao, D.Q., Wang, W.L., Zhang, N.: Hierarchical rough decision theoretic framework for text classification. In: Proceedings of the 9th IEEE International Conference on Cognitive Informatics, pp. 484–489 (2010)Google Scholar
  8. 8.
    Li, W., Huang, Z., Jia, X.: Two-phase classification based on three-way decisions. In: Lingras, P., Wolski, M., Cornelis, C., Mitra, S., Wasilewski, P. (eds.) RSKT 2013. LNCS, vol. 8171, pp. 338–345. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  9. 9.
    Li, H., Zhou, X., Huang, B., Liu, D.: Cost-sensitive three-way decision: a sequential strategy. In: Lingras, P., Wolski, M., Cornelis, C., Mitra, S., Wasilewski, P. (eds.) RSKT 2013. LNCS, vol. 8171, pp. 325–337. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  10. 10.
    Li, H.X., Zhou, X.Z., Zhao, J.B., Liu, D.: Non-monotonic attribute reduction in decision-theoretic rough sets. Fundamenta Informaticae 126(4), 415–432 (2013)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Liu, D., Liang, D.: An overview of function based three-way decisions. In: Miao, D., Pedrycz, W., Slezak, D., Peters, G., Hu, Q., Wang, R. (eds.) RSKT 2014. LNCS, vol. 8818, pp. 812–823. Springer, Heidelberg (2014) Google Scholar
  12. 12.
    Liu, D., Liang, D.C., Wang, C.C.: A novel three-way decision model based on incomplete information system. Knowl.-Based Syst. (2015). doi: 10.1016/j.knosys.2015.07.036CrossRefGoogle Scholar
  13. 13.
    Sokolova, M., Lapalme, G.: A systematic analysis of performance measures for classification tasks. Inf. Process. Manag. 45, 427–437 (2009)CrossRefGoogle Scholar
  14. 14.
    Yao, Y.Y.: Three-way decisions with probabilistic rough sets. Inf. Sci. 180(3), 341–353 (2010)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Yao, Y.Y.: The superiority of three-way decisions in probabilistic rough set models. Inf. Sci. 181(6), 1080–1096 (2011)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Yao, J.T., Azam, N.: Web-based medical decision support systems for three-way medical decision making with game-theoretic rough sets. IEEE Trans. Fuzzy Syst. 23(1), 3–15 (2015)CrossRefGoogle Scholar
  17. 17.
    Yu, H., Zhang, C., Hu, F.: An incremental clustering approach based on three-way decisions. In: Cornelis, C., Kryszkiewicz, M., Ślȩzak, D., Ruiz, E.M., Bello, R., Shang, L. (eds.) RSCTC 2014. LNCS, vol. 8536, pp. 152–159. Springer, Heidelberg (2014) Google Scholar
  18. 18.
    Zhou, B., Yao, Y.Y., Luo, J.G.: Cost-sensitive three-way email spam filtering. J. Intell. Inf. Syst. 42(1), 19–45 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (, which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  1. 1.School of Computer Science and EngineeringNanjing University of Science and TechnologyNanjingChina
  2. 2.State Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina

Personalised recommendations