Automated Text Categorization: The Two-Dimensional Probabilistic Model

  • Giorgio Maria Di Nunzio
Part of the The Information Retrieval Series book series (INRE, volume 22)


In Automated Text Categorization (ATC), a general inductive process automatically builds a classifier for the categories involved in the process by observing the properties of a set of pre-classified documents; from these properties, the inductive process learns the characteristics that a new unseen document should have in order to be categorized under a specific category. Probabilistic models, such as the Naïve Bayes Models (NBs), achieve a performance comparable to more sophisticated models, and they prove to be very efficient

We present the probabilistic model named Two-Dimensional Probabilistic Model (2DPM) which starts from different hypotheses from those of the NB models: instead of independent events, terms are seen as disjoint events, and documents are represented as the union of these events. The set of probability measures defined in this model work in such a way that a reduction of the vocabulary of terms in order to reduce the complexity of the problem is ultimately not necessary. Moreover, the model defines a direct relationship between the probability of a document given a category of interest and a point on a two-dimensional space. In this light, it is possible to graph entire collections of documents on a Cartesian plane, and to design algorithms that categorize documents directly on this two-dimensional representation. This graphical representation has been useful to give insights into the development of the theoretical aspects of the 2DPM. Experiments on traditional test collections for ATC show that the 2DPM performs with a greater degree of statistical significance than the multinomial NB model


automated text categorization probabilistic models visualization of large data sets 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Damerau, F., Zhang, T., Weiss, S., Indurkhya, N.: Text Categorization for a Comprehensive Time-Dependent Benchmark. Information Processing and Management 40(2), 209–221 (2004)zbMATHCrossRefGoogle Scholar
  2. 2.
    Debole, F., Sebastiani, F.: An Analysis of the Relative Hardness of Reuters–21578 Subsets. Journal of the American Society for Information Science and Technology 56(6), 584–596 (2005)CrossRefGoogle Scholar
  3. 3.
    Di Nunzio, G.M.: A Bidimensional View of Documents for Text Categorization. In: S. McDonald, J. Tait (eds.) Proceedings of the 26th European Conference on Information Retrieval (ECIR 2004), Lecture Notes in Computer Science (LNCS), Vol. 2997, pp. 112–126. Springer, Heidelberg, Germany (2004)Google Scholar
  4. 4.
    Di Nunzio, G.M.: Cross–comparison for Two-dimensional Text Categorization. In: A. Apostolico, M. Melucci (eds.) Proceedings of the 11th Symposium on String Processing and Information Retrieval (SPIRE 2004), Lecture Notes in Computer Science (LNCS), Vol. 3246, pp. 125–126. Springer, Heidelberg, Germany (2004)Google Scholar
  5. 5.
    Di Nunzio, G.M.: 3–D Environment to Represent Textual Documents for Duplicate Detection and Collection Examination. In: T. Catarci, S. Christodoulakis, A. Del Bimbo (eds.) Seventh International Workshop of the EU Network of Excellence DELOS on Audio-Visual Content and Information Visualization in Digital Libraries (AVIVDiLib-05), pp. 12–21. Centromedia, Capannori (Lucca), Italy, Cortona (AR), Italy (2005)Google Scholar
  6. 6.
    Di Nunzio, G.M.: Visualization and Classification of Documents: A New Probabilistic Model to Automated Text Classification. Bulletin of the IEEE Technical Committee on Digital Libraries (IEEE-TCDL) 2(2) (2006)Google Scholar
  7. 7.
    Di Nunzio, G.M., Micarelli, A.: Does a New Simple Gaussian Weighting Approach Perform Well in Text Categorization? In: G. Gottlob, T. Walsh (eds.) Proceedings of the 18th International Joint Conference of Artificial Intelligence (IJCAI 2003), pp. 581–586. Morgan Kaufmann, San Francisco, CA, USA (2003)Google Scholar
  8. 8.
    Di Nunzio, G.M., Micarelli, A.: Pushing “Underfitting” to the Limit: Learning in Bidimensional Text Categorization. In: R.L. de Mántaras, L. Saitta (eds.) Proceedings of the 16th Eureopean Conference on Artificial Intelligence (ECAI 2004), including Prestigious Applicants of Intelligent Systems (PAIS 2004), pp. 465–469. IOS Press (2004)Google Scholar
  9. 9.
    Dumais, S.: LSI meets TREC: A Status Report. In: G.B. Lamont, H. Haddad, G.A. Papadopoulos, B. Panda (eds.) Proceedings of 2003 ACM Symposium on Applied Computing (SAC 2003), pp. 137–152. ACM Press, New York, USA (2003)Google Scholar
  10. 10.
    Ferreira de Oliveria, M., Levkowitz, H.: From Visual Data Exploration to Visual Data Mining: A Survey. IEEE Transactions on Visualization and Computer Graphics 9(3), 378–394 (2003)CrossRefGoogle Scholar
  11. 11.
    Fuhr, N.: Models in Information Retrieval. In: M. Agosti, F. Crestani, G. Pasi (eds.) Lectures on Information Retrieval, Lecture Notes in Computer Science (LNCS), Vol. 1980, pp. 21–50. Springer, Heidelberg, Germany (2000)Google Scholar
  12. 12.
    Krowne, A., Halbert, M.: An Initial Evaluation of Automated Organization for Digital Library Browsing. In: M. Marlino, T. Sumner, F. Shipman III (eds.) ACM/IEEE Joint Conference on Digital Libraries (JCDL 2005), pp. 246–255. ACM Press, New York, NY, USA, Denver, CO, USA (2005)CrossRefGoogle Scholar
  13. 13.
    Lewis, D.: Naive (Bayes) at Forty: The Independence Assumption in Information Retrieval. In: C. Nedellec, C. Rouveirol (eds.) Machine Learning: ECML-98, 10th European Conference on Machine Learning, Lecture Notes in Computer Science (LNCS), Vol. 1398, pp. 4–15. Springer, Heidelberg, Germany (1998)CrossRefGoogle Scholar
  14. 14.
    Mladenič, D., Brank, J., Grobelnik, M., Milic-Frayling, N.: Feature Selection using Linear Classifier Weights: Interaction with Classification Models. In: J. Callan, N. Fuhr, W. Nejdl (eds.) Peer-to-Peer Information Retrieval, pp. 234–241. ACM Press, New York, NY, USA (2004)Google Scholar
  15. 15.
    Poulin, B., Eisner, R., Szafron, D., Lu, P., Greiner, R., Wishart, D.S., Fyshe, A., Pearcy, B., Macdonell, C., Anvik, J.: Visual Explanation of Evidence with Additive Classifiers. In: Proceedings of the 21st National Conference on Artificial Intelligence (AAAI) and the 18th Innovative Applications of Artificial Intelligence (IAAI) Conference, July 16-20, 2006, Boston, Massachusetts, USA. AAAI Press (2006)Google Scholar
  16. 16.
    Rifkin, R., Klautau, A.: In Defense of One-Vs-All Classification. Journal of Machine Learning Research 5, 101–141 (2004)MathSciNetGoogle Scholar
  17. 17.
    Sebastiani, F.: Machine Learning in Automated Text Categorization. ACM Computing Surveys 34(1), 1–47 (2002)CrossRefGoogle Scholar
  18. 18.
    Van Rijsbergen, C.: Information Retrieval. Butterworth-Heinemann, Newton, MA, USA (1979)Google Scholar
  19. 19.
    Webb, G., Boughton, J., Wang, Z.: Not So Naive Bayes: Aggregating One-Dependence Estimators. Machine Learning 58(1), 5–24 (2005)zbMATHCrossRefGoogle Scholar
  20. 20.
    Yang, Y., Pedersen, J.: A Comparative Study on Feature Selection in Text Categorization. In: D. Fisher (ed.) Proceedings of the Fourteenth International Conference on Machine Learning (ICML 1997), pp. 412–420. Morgan Kaufmann Publishers, San Francisco, CA, USA, Nashville, Tennessee, USA (1997)Google Scholar
  21. 21.
    Yang, Y., Zhang, J., Kisiel, B.: A Scalability Analysis of Classifiers in Text Categorization. In: C. Clarke, G. Cormack, J. Callan, D. Hawking, A. Smeaton (eds.) Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2003), pp. 96–103. ACM Press, New York, USA (2003). Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Giorgio Maria Di Nunzio
    • 1
  1. 1.Department of Information EngineeringUniversity of PaduaVia Gradenigo 6/aItaly

Personalised recommendations