Abstract
Text categorisation in commercial application poses several limiting constraints on the technology solutions to be employed. This paper describes how a method with some potential improvements is evaluated for practical purposes and argues for a richer and more expressive evaluation procedure. In this paper one such method is exemplified by a precision-recall matrix which sacrifices convenience for expressiveness.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Barak, L., Dagan, I., Shnarch, E.: Text categorization from category name via lexical reference. In: Proceedings of Human Language Technologies: Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 33–36. Association for Computational Linguistics (2009)
Gliozzo, A., Strapparava, C., Dagan, I.: Improving text categorization bootstrapping via unsupervised learning. ACM Trans. Speech Lang. Process. (TSLP) 6(1), 1 (2009)
Ko, Y., Seo, J.: Text classification from unlabeled documents with bootstrapping and feature projection techniques. Inf. Process. Manage. 45(1), 70–83 (2009)
Liebeskind, C., Kotlerman, L., Dagan, I.: Text categorization from category name in an industry-motivated scenario. Lang. Resour. Eval. 49(2), 227–261 (2015)
McCallum, A., Nigam, K., Rennie, J., Seymore, K.: A machine learning approach to building domain-specific search engines. In: IJCAI, vol. 99, pp. 662–667. Citeseer (1999)
Qiu, Q., Zhang, Y., Zhu, J., Qu, W.: Building a text classifier by a keyword and wikipedia knowledge. In: Huang, R., Yang, Q., Pei, J., Gama, J., Meng, X., Li, X. (eds.) ADMA 2009. LNCS, vol. 5678, pp. 277–287. Springer, Heidelberg (2009)
Sahlgren, M., Gyllensten, A.C., Espinoza, F., Hamfors, O., Holst, A., Karlgren, J., Olsson, F., Persson, P., Viswanathan, A.: The Gavagai living lexicon. In: 10th Language Resources and Evaluation Conference, Portoroz (2016)
Schohn, G., Cohn, D.: Less is more: active learning with support vector machines. In: ICML, pp. 839–846. Citeseer (2000)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Karlsson, V., Herman, P., Karlgren, J. (2016). Evaluating Categorisation in Real Life – An Argument Against Simple but Impractical Metrics. In: Fuhr, N., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2016. Lecture Notes in Computer Science(), vol 9822. Springer, Cham. https://doi.org/10.1007/978-3-319-44564-9_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-44564-9_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-44563-2
Online ISBN: 978-3-319-44564-9
eBook Packages: Computer ScienceComputer Science (R0)