Advertisement

Document Classification with Unsupervised Artificial Neural Networks

  • Dieter Merkl
  • Andreas Rauber
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 50)

Abstract

Text collections may be regarded as an almost perfect application arena for unsupervised neural networks. This is because many operations computers have to perform on text documents are classification tasks based on noisy patterns. In particular we rely on self-organizing maps which produce a map of the document space after their training process. From geography, however, it is known that maps are not always the best way to represent information spaces. For most applications it is better to provide a hierarchical view of the underlying data collection in form of an atlas where, starting from a map representing the complete data collection, different regions are shown at finer levels of granularity. Using an atlas, the user can easily “zoom” into regions of particular interest while still having general maps for overall orientation. We show that a similar display can be obtained by using hierarchical feature maps to represent the contents of a document archive. These neural networks have a layered architecture where each layer consists of a number of individual self-organizing maps. By this, the contents of the text archive may be represented at arbitrary detail while still having the general maps available for global orientation.

Keywords

Artificial Neural Network Information Retrieval Weight Vector Artificial Neural Network Model Input Pattern 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bayer, T., Renz, L, Stein, M., and Kressel, U. (1996). Domain and language independent feature extraction for Statistical text categorization. In Proc of the Workshop on Language Engineering for Document Analysis and Recognition, Sussex, United Kingdom.Google Scholar
  2. 2.
    Belew, R. K. (1987). A connectionist approach to conceptual information retrieval. In Proc of the Infi Conference on Artificial Intelligence and Law (ICAIL ’87), Boston, MA.Google Scholar
  3. 3.
    Belew, R. K. (1989). Adaptive information retrieval: Using a connectionist representation to retrieve and learn about documents. In Proc ofthe ACM SIGIR Int’l Conf on Research and Development in Information Retrieval (SIGIR’89), Cambridge, MA.Google Scholar
  4. 4.
    Bishop, C. M., Svensen, M., and Williams, C. K. I. (1996a). GTM: A principled alternative to the self-organizing map. In Proc of the Int’l Conf on Artificial Neural Networks (ICANN’96), Bochum, Germany.Google Scholar
  5. 5.
    Bishop, C. M., Svensen, M., and Williams, C. K. I. (1996b). GTM: The generative topographic mapping. Technical Report NCRG/96/015, Aston University, Neural Computing Research Group, http://www.ncrg.aston.ac.uk, Birmingham, United Kingdom.Google Scholar
  6. 6.
    Carpenter, G. A. and Grossberg, S. (1988). The ART of adaptive pattern recognition by a self-organizing neural network. IEEE Computer, 21(3).Google Scholar
  7. 7.
    Cottrell, M. and Fort, J.-C. (1987). Etüde d’un processus d’auto-organisation. Annales de l’Institut Henri Poincare, 23(1).Google Scholar
  8. 8.
    Cottrell, M., Fort, J.-C, and Pages, G. (1994). Two or three things that we know about the Kohonen algorithm. In Proc of the European Symposium on Artificial Neural Networks (ESANN’94), Bruxelles, Belgium.Google Scholar
  9. 9.
    Crestani, F. (1993). Learning strategies for an adaptive information retrieval system using neural networks. In Proc of the IEEE Int’l Conf on Neural Networks (ICNN’93), San Francisco, California.Google Scholar
  10. 10.
    Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., and Hashman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6).Google Scholar
  11. 11.
    Fritzke, B. (1994). Growing Cell Structures: A self-organizing network for unsupervised and supervised learning. Neural Networks, 7(9).Google Scholar
  12. 12.
    Hearst, M. A. and Pedersen, J. O. (1996). Reexamining the Cluster hypothesis: Scatter/Gather on retrieval results. In Proc Infi ACM SIGIR Conf on R&D in Information Retrieval (SIGIR’96), Zürich, Switzerland.Google Scholar
  13. 13.
    Honkela, T. (1997). Self-organizing maps of words for natural language processing applications. In Proceedings International ICSC Symposium on Soft Computing, Nimes, France.Google Scholar
  14. 14.
    Honkela, T., Kaski, S., Lagus, K., and Kohonen, T. (1996). Newsgroup exploration with WEBSOM method and browsing interface. Technical Report A32, Helsinki University of Technology, Laboratory of Computer and Information Science, Espoo, Finland.Google Scholar
  15. 15.
    Honkela, T., Kaski, S., Lagus, K., and Kohonen, T. (1997). WEBSOM — selforganizing maps of document collections. In Proceedings Workshop on Seif- Organizing Maps, Espoo, Finland.Google Scholar
  16. 16.
    Honkela, T., Pulkki, V., and Kohonen, T. (1995). Contextual relations of words in Grimm tales analyzed by self-organizing maps. In Proc of the Infi Conf on Artificial Neural Networks (ICANN’95), Paris, France.Google Scholar
  17. 17.
    Jolliffe, I. T. (1986). Principal Component Analysis. Springer-Verlag, Berlin.CrossRefGoogle Scholar
  18. 18.
    Kandel, E. R., Siegelbaum, S. A., and Schwartz, J. H. (1991). Synaptic transmission. In Kandel, E. R., Schwartz, J. H., and Jessell, T. M., editors, Principles of Neural Science. Elsevier, New York.Google Scholar
  19. 19.
    Keane, S., Ratnaike, V., and Wilkinson, R. (1996). Hierarchical news filtering. In Proc ofthe Infi Conf on Practical Aspects of Knowledge Management, Basel, Switzerland.Google Scholar
  20. 20.
    Kohle, M. and Merkl, D. (1996). Visualizing similarities in high dimensional input Spaces with a growing and Splitting neural network. In Proc of the Infi Conf on Artificial Neural Networks (ICANN’96), Bochum, Germany.Google Scholar
  21. 21.
    Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43.Google Scholar
  22. 22.
    Kohonen, T. (1995). Self-organizing maps. Springer-Verlag, Berlin.CrossRefGoogle Scholar
  23. 23.
    Kohonen, T. (1998). Self-organization of very large document collections: State of the art. In Proc of the Infi Conf on Artificial Neural Networks (ICANN’98), Skövde, Sweden.Google Scholar
  24. 24.
    Kohonen, T., Kaski, S., Lagus, K., and Honkela, T. (1996). Very large two-level SOM for the browsing of newsgroups. In Proc of the Infi Conf on Artificial Neural Networks (ICANN’96), Bochum, Germany.Google Scholar
  25. 25.
    Lagus, K., Honkela, T., Kaski, S., and Kohonen, T. (1996). Self-organizing maps of document collections: A new approach to interactive exploration. In Proc of the Infi Conf on Knowledge Discovery and Data Mining (KDD-96), Portland, OR.Google Scholar
  26. 26.
    Lin, X., Soergel, D., and Marchionini, G. (1991). A self-organizing semantic map for information retrieval. In Proc of the ACM SIGIR Infi Conf on Research and Development in Information Retrieval (SIGIR’91), Chicago, IL.Google Scholar
  27. 27.
    Merkl, D. (1995a). A connectionist view on document Classification. In Proc of the Australasian Database Conf (ADC’95), Adelaide, SA.Google Scholar
  28. 28.
    Merkl, D. (1995b). Content-based document Classification with highly compressed input data. In Proc of the Infi Conf on Artificial Neural Networks (ICANN’95), Paris, France.Google Scholar
  29. 29.
    Merkl, D. (1995c). Content-based Software Classification by self-organization. In Proc of the IEEE Infi Conf on Neural Networks (ICNN’95), Perth, WA.Google Scholar
  30. 30.
    Merkl, D. (1995d). The effect of lateral Inhibition on learning speed and precision of a self-organizing map. In Proc of the Australiern, Conf on Neural Networks, Sydney, NSW.Google Scholar
  31. 31.
    Merkl, D. (1997a). Exploration of document collections with self-organizing maps: A novel approach to similarity representation. In Proc of the European Symposium on Principles of Data Mining and Knowledge Discovery (PKDD’97), Trondheim, Norway.Google Scholar
  32. 32.
    Merkl, D. (1997b). Exploration of text collections with hierarchical feature maps. In Proc Infi ACM SIGIR Conf on R&D in Information Retrieval (SI-GIR’97), Phüadelphia, PA.Google Scholar
  33. 33.
    Merkl, D. (1998). Text Classification with self-organizing maps: Some lessons learned. Neurocomputing, 21(1–3).Google Scholar
  34. 34.
    Merkl, D. and Rauber, A. (1999). Uncovering associations between documents. In Proc of the IJCAF99 Workshop on Text Mining, Stockholm, Sweden.Google Scholar
  35. 35.
    Merkl, D., Schweighofer, E., and Winiwarter, W. (1994). CONCAT: Connotation analysis of thesauri based on the interpretation of context meaning. In Proc of the Infi Conference on Database and Expert Systems Applications (DEXA ‘94), Athens, Greece.Google Scholar
  36. 36.
    Miikkulainen, R. (1990). Script recognition with hierarchical feature maps. Connection Science, 2.Google Scholar
  37. 37.
    Miikkulainen, R. (1991). Self-organizing process based on lateral inhibition and synaptic resource redistribution. In Proc of the Infi Conf on Artificial Neural Networks (ICANN’91), Espoo, Finland.Google Scholar
  38. 38.
    Miikkulainen, R. (1993). Subsymbolic Natural Language Processing: An integrated model of Scripts, lexicon, and memory. MIT-Press, Cambridge, MA.Google Scholar
  39. 39.
    Rauber, A. and Merkl, D. (1998). Creating an order in distributed digital libraries by integrating independent self-organizing maps. In Proc of the Infi Conf on Artificial Neural Networks (ICANN’98), Skövde, Sweden.Google Scholar
  40. 40.
    Rauber, A. and Merkl, D. (1999). Automatic labeling of self-organizing maps: Making a treasure-map reveal its secrets. In Proc of the Pacific Asia Conf on Knowledge Discovery and Data Mining (PAKDD’99), Beijing, China.Google Scholar
  41. 41.
    Ritter, H. and Kohonen, T. (1989). Self-organizing semantic maps. Biological Cybernetics, 61.Google Scholar
  42. 42.
    Rose, D. E. (1994). A Symbolic and Connectionist Approach to Legal Information Retrieval. Lawrence Erlbaum, Hillsdale.Google Scholar
  43. 43.
    Rose, D. E. and Belew, R. K. (1989). Legal information retrieval: A hybrid approach. In Proc of the Infi Conference on Artificial Intelligence and Law (ICAIL’89), Vancouver, Canada.Google Scholar
  44. 44.
    Roussinov, D. and Ramsey, M. (1998). Information forage through adaptive visualization. In Proc of the ACM Infi Conf on Digital Libraries (DL’98), Pittsburgh, PA.Google Scholar
  45. 45.
    Salton, G. (1989). Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley, Reading, MA.Google Scholar
  46. 46.
    Salton, G. and Buckley, C. (1988). Term weighting approaches in automatic text retrieval. Information Processing & Management, 24(5).Google Scholar
  47. 47.
    Turtle, H. R. and Croft, W. B. (1992). A comparison of text retrieval models. Computer Journal, 35(3).Google Scholar
  48. 48.
    Wilkinson, R. and Hingston, P. (1991). Incorporating the vector space model in a neural network used for Information retrieval. In Proc of the A CM SIGIR Infi Conf on Research and Development in Information Retrieval (SIGIR’91), Chicago, IL.Google Scholar
  49. 49.
    Willet, P. (1988). Recend trends in hierarchic document clustering: A critical review. Information Processing & Management, 24.Google Scholar
  50. 50.
    Zavrel, J. (1996). Neural navigation interfaces for information retrieval: Are they more than an appealing idea? Artificial Intelligence Review, 10.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Dieter Merkl
    • 1
  • Andreas Rauber
    • 1
  1. 1.Institut für SoftwaretechnikTechnische Universität WienWienAustria

Personalised recommendations