ClasSi: Measuring Ranking Quality in the Presence of Object Classes with Similarity Information

  • Anca Maria Ivanescu
  • Marc Wichterich
  • Thomas Seidl
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7104)


The quality of rankings can be evaluated by computing their correlation to an optimal ranking. State of the art ranking correlation coefficients like Kendall’s τ and Spearman’s ρ do not allow for the user to specify similarities between differing object classes and thus treat the transposition of objects from similar classes the same way as that of objects from dissimilar classes. We propose ClasSi, a new ranking correlation coefficient which deals with class label rankings and employs a class distance function to model the similarities between the classes. We also introduce a graphical representation of ClasSi akin to the ROC curve which describes how the correlation evolves throughout the ranking.


ranking quality measure class similarity ClasSi 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ferri, C., Hernández-Orallo, J., Salido, M.A.: Volume under the ROC surface for multi-class problems. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) ECML 2003. LNCS (LNAI), vol. 2837, pp. 108–120. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  2. 2.
    Flach, P., Blockeel, H., Ferri, C., Hernández-Orallo, J., Struyf, J.: Decision support for data mining; introduction to ROC analysis and its applications. In: Data Mining and Decision Support: Integration and Collaboration, pp. 81–90. Kluwer Academic Publishers (2003)Google Scholar
  3. 3.
    Goodman, L.A., Kruskal, W.H.: Measures of association for cross classifications. Journal of the American Statistical Association 49(268), 732–764 (1954)zbMATHGoogle Scholar
  4. 4.
    Hand, D.J., Till, R.J.: A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine Learning 45, 171–186 (2001)CrossRefzbMATHGoogle Scholar
  5. 5.
    Hassan, M.R., Ramamohanarao, K., Karmakar, C.K., Hossain, M.M., Bailey, J.: A Novel Scalable Multi-Class ROC for Effective Visualization and Computation. In: Zaki, M.J., Yu, J.X., Ravindran, B., Pudi, V. (eds.) PAKDD 2010. LNCS, vol. 6118, pp. 107–120. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Kendall, M.: A new measure of rank correlation. Biometrika 30(1-2), 81–89 (1938)CrossRefzbMATHGoogle Scholar
  7. 7.
    Kendall, M., Gibbons, J.D.: Rank Correlation Methods. Edward Arnold (1990)Google Scholar
  8. 8.
    Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press (2008)Google Scholar
  9. 9.
    van Rijsbergen, C.J.: Information Retrieval, 2nd edn. Butterworths, London (1979)zbMATHGoogle Scholar
  10. 10.
    Somers, R.H.: A new asymmetric measure of association for ordinal variables. American Sociological Review 27(6), 799–811 (1962)CrossRefGoogle Scholar
  11. 11.
    Spearman, C.: The proof and measurement of association between two things. The American Journal of Psychology 100, 441–471 (1987)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Anca Maria Ivanescu
    • 1
  • Marc Wichterich
    • 1
  • Thomas Seidl
    • 1
  1. 1.Data Management and Data Exploration GroupRWTH Aachen UniversityAachenGermany

Personalised recommendations