Abstract
Independent Component Analysis (ICA) is an essential building block for data analysis in many applications. Selecting the truly meaningful components from the result of an ICA algorithm, or comparing the results of different algorithms, however, are non-trivial problems. We introduce a very general technique for evaluating ICA results rooted in information-theoretic model selection. The basic idea is to exploit the natural link between non-Gaussianity and data compression: The better the data transformation represented by one or several ICs improves the effectiveness of data compression, the higher is the relevance of the ICs. In an extensive experimental evaluation we demonstrate that our novel information-theoretic measure robustly selects the most interesting components from data without requiring any assumptions or thresholds.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barron, A.R., Rissanen, J., Yu, B.: The minimum description length principle in coding and modeling. IEEE Transactions on Information Theory 44(6), 2743 (1998)
Pelleg, D., Moore, A.: X-means: Extending K-means with efficient estimation of the number of clusters. In: ICML Conference, pp. 727–734 (2000)
Lee, T.C.M.: Regression spline smoothing using the minimum description length principle. Statistics and Probability Letters 48, 71–82 (2000)
Böhm, C., Faloutsos, C., Plant, C.: Outlier-robust clustering using independent components. In: SIGMOD Conference, pp. 185–198 (2008)
Lee, T.W., Lewicki, M.S.: The generalized gaussian mixture model using ICA. In: ICA Conference (2000)
Hyvärinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks 10(3), 626–634 (1999)
Wismüller, A., Lange, O., Dersch, D.R., Leinsinger, G.L., Hahn, K., Pütz, B., Auer, D.: Cluster analysis of biomedical image time-series. Int. J. Comput. Vision 46(2), 103–128 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Plant, C., Theis, F.J., Meyer-Baese, A., Böhm, C. (2010). Information-Theoretic Model Selection for Independent Components. In: Vigneron, V., Zarzoso, V., Moreau, E., Gribonval, R., Vincent, E. (eds) Latent Variable Analysis and Signal Separation. LVA/ICA 2010. Lecture Notes in Computer Science, vol 6365. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15995-4_32
Download citation
DOI: https://doi.org/10.1007/978-3-642-15995-4_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15994-7
Online ISBN: 978-3-642-15995-4
eBook Packages: Computer ScienceComputer Science (R0)