Abstract
We propose a feature selection approach for clustering which extends Koller and Sahami’s mutual-information-based criterion to the unsupervised case. This is achieved with the help of a mixture-based model and the corresponding expectation-maximization algorithm. The result is a backward search scheme, able to sort the features by order of relevance. Finally, an MDL criterion is used to prune the sorted list of features, yielding a feature selection criterion. The proposed approach can be classified as a wrapper, since it wraps the mixture estimation algorithm in an outer layer that performs feature selection. Preliminary experimental results show that the proposed method has promising performance.
Work partially supported by the Foundation for Science and Technology (Portugal), grant POSI/33143/SRI/2000, and the Office of Naval Research (USA), grant 00014- 01-1-0266.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Koller, D., Sahami, M.: Toward optimal feature selection. In: Proceedings of the International Conference on Machine Learning, Bari, Italy, pp. 284–292 (1996)
Kohavi, R., John, G.: Wrappers for feature subset selection. Artificial Intelligence 97(1-2), 273–324 (1997)
Jain, A.K., Zongker, D.: Feature selection: Evaluation, application, and small sample performance. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(2), 153–158 (1997)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2001)
Jain, A.K., Dubes, R.: Algorithms for Clustering Data. Prentice Hall, Englewood Cliffs (1988)
Fraley, C., Raftery, A.: Model based clustering, discriminant analysis, and density estimation. Journal of the American Statist. Assoc. 97, 611–631 (2002)
McLachlan, G., Basford, K.: Mixture Models: Inference and Application to Clustering. Marcel Dekker, New York (1988)
McLachlan, G., Peel, D.: Finite Mixture Models. John Wiley & Sons, New York (2000)
Figueiredo, M., Jain, A.K.: Unsupervised learning of finite mixture models. IEEE Trans. on Pattern Analysis and Machine Intelligence 24, 381–396 (2002)
Dy, J., Brodley, C.: Feature subset selection and order identification for unsupervised learning. In: Proc. 17th International Conf. on Machine Learning, pp. 247–254. Morgan Kaufmann, San Francisco (2000)
Vaithyanathan, S., Dom, B.: Generalized model selection for unsupervised learning in high dimensions. In: Solla, S., Leen, T., Müller, K.R. (eds.) Advances in Neural Information Processing Systems 12. MIT Press, Cambridge (2000)
Dash, M., Liu, H.: Feature selection for clustering. In: Proc. of Pacific-Asia Conference on Knowledge Discovery and Data Mining (2000)
Kim, Y., Street, W., Menczer, F.: Feature Selection in Unsupervised Learning via Evolutionary Search. In: Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2000)
Devaney, M., Ram, A.: Efficient feature selection in conceptual clustering. In: International Conference on Machine Learning, pp. 92–97 (1997)
Dempster, A., Laird, N., Rubin, D.: Maximum likelihood estimation from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B 39, 1–38 (1977)
McLachlan, G., Krishnan, T.: The EM Algorithm and Extensions. John Wiley & Sons, New York (1997)
Cover, T., Thomas, J.: Elements of Information Theory. John Wiley & Sons, New York (1991)
Trunk, G.: A problem of dimensionality: A simple example. IEEE Trans. on Pattern Analysis and Machine Intelligence 1(3), 306–307 (1979)
Rissanen, J.: Stochastic Complexity in Stastistical Inquiry. World Scientific, Singapore (1989)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Figueiredo, M.A.T., Jain, A.K., Law, M.H. (2003). A Feature Selection Wrapper for Mixtures. In: Perales, F.J., Campilho, A.J.C., de la Blanca, N.P., Sanfeliu, A. (eds) Pattern Recognition and Image Analysis. IbPRIA 2003. Lecture Notes in Computer Science, vol 2652. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-44871-6_27
Download citation
DOI: https://doi.org/10.1007/978-3-540-44871-6_27
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40217-6
Online ISBN: 978-3-540-44871-6
eBook Packages: Springer Book Archive