LIMBO: Scalable Clustering of Categorical Data

  • Periklis Andritsos
  • Panayiotis Tsaparas
  • Renée J. Miller
  • Kenneth C. Sevcik
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2992)


Clustering is a problem of great practical importance in numerous applications. The problem of clustering becomes more challenging when the data is categorical, that is, when there is no inherent distance measure between data values. We introduce LIMBO, a scalable hierarchical categorical clustering algorithm that builds on the Information Bottleneck (IB) framework for quantifying the relevant information preserved when clustering. As a hierarchical algorithm, LIMBO has the advantage that it can produce clusterings of different sizes in a single execution. We use the IB framework to define a distance measure for categorical tuples and we also present a novel distance measure for categorical attribute values. We show how the LIMBO algorithm can be used to cluster both tuples and values. LIMBO handles large data sets by producing a memory bounded summary model for the data. We present an experimental evaluation of LIMBO, and we study how clustering quality compares to other categorical clustering algorithms. LIMBO supports a trade-off between efficiency (in terms of space and time) and quality. We quantify this trade-off and demonstrate that LIMBO allows for substantial improvements in efficiency with negligible decrease in quality.


Execution Time Mutual Information Leaf Node Information Loss Cluster Quality 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andritsos, P., Tsaparas, P., Miller, R.J., Sevcik, K.C.: Limbo: A linear algorithm to cluster categorical data. Technical report, UofT, Dept of CS, CSRG-467 (2003)Google Scholar
  2. 2.
    Andritsos, P., Tzerpos, V.: Software Clustering based on Information Loss Minimization. In: WCRE, Victoria, BC, Canada (2003)Google Scholar
  3. 3.
    Barbará, D., Couto, J., Li, Y.: An Information Theory Approach to Categorical Clustering (submitted for Publication)Google Scholar
  4. 4.
    Barbará, D., Couto, J., Li, Y.: COOLCAT: An entropy-based algorithm for categorical clustering. In: CIKM, McLean, VA (2002)Google Scholar
  5. 5.
    Borodin, A., Roberts, G.O., Rosenthal, J.S., Tsaparas, P.: Finding authorities and hubs from link structures on the World Wide Web. In: WWW-10, Hong Kong (2001)Google Scholar
  6. 6.
    Chiu, T., Fang, D., Chen, J., Wang, Y., Jeris, C.: A Robust and Scalable Clustering Algorithm for Mixed Type Attributes in Large Database Environment. In: KDD, San Francisco, CA (2001)Google Scholar
  7. 7.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley & Sons, Chichester (1991)zbMATHCrossRefGoogle Scholar
  8. 8.
    Barbará, D.: Requirements for Clustering Data Streams. SIGKDD Explorations 3(2) (January 2002)Google Scholar
  9. 9.
    Das, G., Mannila, H.: Context-Based Similarity Measures for Categorical Databases. In: Zighed, D.A., Komorowski, J., Żytkow, J.M. (eds.) PKDD 2000. LNCS (LNAI), vol. 1910, pp. 201–210. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  10. 10.
    Ganti, V., Gehrke, J., Ramakrishnan, R.: CACTUS: Clustering Categorical Data Using Summaries. In: KDD, San Diego, CA (1999)Google Scholar
  11. 11.
    Garey, M.R., Johnson, D.S.: Computers and intractability; a guide to the theory of NP-completeness. W.H. Freeman, New York (1979)zbMATHGoogle Scholar
  12. 12.
    Gibson, D., Kleinberg, J.M., Raghavan, P.: Clustering Categorical Data: An Approach Based on Dynamical Systems. In: VLDB, New York, NY (1998)Google Scholar
  13. 13.
    Guha, S., Rastogi, R., Shim, K.: ROCK: A Robust Clustering Algorithm for Categorical Atributes. In: ICDE, Sydney, Australia (1999)Google Scholar
  14. 14.
    Kleinberg, J.M.: Authoritative Sources in a Hyperlinked Environment. In: SODA, SF, CA (1998)Google Scholar
  15. 15.
    Gluck, M.A., Corter, J.E.: Information, Uncertainty, and the Utility of Categories. In: COGSCI, Irvine, CA, USA (1985)Google Scholar
  16. 16.
    Miller, R.J., Andritsos, P.: On Schema Discovery. IEEE Data Engineering Bulletin 26(3), 39–44 (2003)Google Scholar
  17. 17.
    Slonim, N., Friedman, N., Tishby, N.: Unsupervised Document Classification using Sequential Information Maximization. In: SIGIR, Tampere, Finland (2002)Google Scholar
  18. 18.
    Slonim, N., Tishby, N.: Agglomerative Information Bottleneck. In: NIPS, Breckenridge (1999)Google Scholar
  19. 19.
    Slonim, N., Tishby, N.: Document Clustering Using Word Clusters via the Information Bottleneck Method. In: SIGIR, Athens, Greece (2000)Google Scholar
  20. 20.
    Tishby, N., Pereira, F.C., Bialek, W.: The Information Bottleneck Method. In: 37th Annual Allerton Conference on Communication, Control and Computing, Urban-Champaign, IL (1999)Google Scholar
  21. 21.
    Zhang, T., Ramakrishnan, R., Livny, M.: BIRCH: An efficient Data Clustering Method for Very Large Databases. In: SIGMOD, Montreal, QB (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Periklis Andritsos
    • 1
  • Panayiotis Tsaparas
    • 1
  • Renée J. Miller
    • 1
  • Kenneth C. Sevcik
    • 1
  1. 1.Department of Computer ScienceUniversity of Toronto 

Personalised recommendations