Abstract
We present a new threshold-based clustering algorithm for news articles. The algorithm consists of two phases: in the first, a local optimum of a score function that captures the quality of a clustering is found with an Expectation-Maximization approach. In the second phase, the algorithm reduces the number of clusters and, in particular, is able to build non-spherical–shaped clusters. We also give a mini-batch version which allows an efficient dynamic processing of data points as they arrive in groups. Our experiments on the TDT5 benchmark collection show the superiority of both versions of this algorithm compared to other state-of-the-art alternatives.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Aït-Mokhtar, S., Chanod, J.-P., Roux, C.: Robustness beyond shallowness: incremental deep parsing. Nat. Lang. Eng. 8, 121–144 (2002)
Allan, J., Harding, S., Fisher, D., Bolivar, A., Guzman-Lara, S., Amstutz, P.: Taking Topic Detection From Evaluation to Practice. In: HICSS 2004. IEEE (2004)
Aslam, J.A., Pelekhov, E., Rus, D.: The Star Clustering Algorithm for Static and Dynamic Information Organization. JGAA 8(1), 95–129 (2004)
Dhillon, I.S., Modha, D.S.: Concept decompositions for large sparse text data using clustering. JMLR 42(1), 143–175 (2001)
Ester, M., Kriegel, H.-P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDD 1996, pp. 226–231 (1996)
Frey, B.J., Dueck, D.: Clustering by passing messages between data points. Science 315(5814), 972–976 (2007)
Kastner, I., Monz, C.: Automatic single-document key fact extraction from newswire articles. In: EACL 2009, pp. 415–423 (2009)
van Rijsbergen, C.: Information Retrieval. Butterworths (1979)
Vadrevu, S., Teo, C.H., Rajan, S., Punera, K., Dom, B., Smola, A.J., Chang, Y., Zheng, Z.: Scalable clustering of news search results. In: WSDM 2011, pp. 675–683 (2011)
Yang, Y., Pierce, T., Carbonell, J.: A study of retrospective and on-line event detection. In: SIGIR, pp. 28–36 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gallé, M., Renders, JM. (2012). Full and Mini-batch Clustering of News Articles with Star-EM. In: Baeza-Yates, R., et al. Advances in Information Retrieval. ECIR 2012. Lecture Notes in Computer Science, vol 7224. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28997-2_49
Download citation
DOI: https://doi.org/10.1007/978-3-642-28997-2_49
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28996-5
Online ISBN: 978-3-642-28997-2
eBook Packages: Computer ScienceComputer Science (R0)