Abstract
Overcomplete representations have been advocated because they allow a basis to better approximate the underlying statistical density of the data which can lead to representations that better capture the underlying structure in the data. The prior distributions for the coefficients of these models, however, are assumed to be fixed, not adaptive to the data, and hereby inaccurate. Here we describe a method for learning overcomplete representations with a generalized Gaussian prior, which can fit a broader range of statistical distributions by varying the value of the steepness parameter β. Using this distribution in overcomplete representations, empirical results were obtained for the blind source separation of more sources than mixtures, which show that the accuracy of the density estimation is improved.
Supported by the National Natural Science Foundation of China under Grant Nos. 60373029 and the National Research Foundation for the Doctoral Program of Higher Education of China under Grant Nos. 20050004001.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Lewicki, M.S., Sejnowski, T.J.: Learning Overcomplete Representations. Neural Computation 12(2), 337–365 (2000)
Lewicki, M.S., Olshausen, B.A.: A Probabilistic Framework for the Adaptation and Comparison of Image Codes. J. Opt. Soc. Am. A: Optics, Image Science, and Vision 16(7), 1587–1601 (1999)
Lewicki, M.S., Olshausen, B.A.: Inferring Sparse, Overcomplete Image Codes Using an Effiecient Coding Framework. In: Advances in Neural and Information Processing Systems, vol. 10. Morgan Kaufmann, San Mateo (1998)
Olshausen, B.A., Field, D.J.: Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1? Visual Research 37(33), 11–25 (1997)
Hyvärinen, A., Oja, E.: Independent Component Analysis: Algorithms and Application. Neural Networks 13(4-5), 411–430 (2000)
Comon, P.: Independent Component Analysis—A new concept? Signal Processing 36(3), 287–314 (1994)
: The Generalized Gaussian Mixture Model Using ICA. In: International Workshop on Independent Component Analysis, pp. 239–244. Helsinki (2000)
Lee, T.W., Lewicki, M.S., Sejnowski, T.J.: ICA Mixture Models for Unsupervised Classification of Non-Gaussian Sources and Automatic Context Switching in Blind Signal Separation. IEEE Transactions on Pattern Analysis and Machine intelligence 22(10), 1078–1089 (2000)
Amari, S.: Neural Learning in Structured Parameters Spaces. In: Advances in Neural Information Processing Systems, vol. 10. MIT Press, Cambridge (1997)
Amari, S.: Natural Gradient Works Efficiently in Learning. Neural Computation 10, 251–276 (1998)
Olshausen, B.A.: Principles of Image Representation in Visual Cortex. In: Chalupa, L.M., Werner, J.S. (eds.) The Visual Neurosciences. MIT Press, Cambridge (2002)
Olshausen, B.A., Field, D.J.: Sparse Coding of Sensory Inputs. Current Opinion in Neurobiology 14, 481–487 (2004)
Foldiak, P., Young, M.P.: Sparse Coding in Primate Cortex. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks, 2nd edn. The MIT Press, Cambridge (2002)
Olshausen, B.A., Millman, K.J.: Learning Sparse Codes with a Mixture-of-Gaussian Prior. In: Solla, S.A., Leen, T.K., Müller, K.-R. (eds.) Advances in Neural Information Processing Systems, vol. 12. MIT press, Cambridge (2000)
Lee, T.W., Lewicki, M.S., Sejnowski, T.J.: ICA Mixture Models for Unsupervised Classification of non-Gaussian Classes and Automatic Context Switching in Blind Signal Separation. IEEE Transactions on Pattern Recognition Machine Intelligence 22(10), 1078–1089 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liao, LZ., Luo, SW., Tian, M., Zhao, LW. (2006). Learning Overcomplete Representations with a Generalized Gaussian Prior. In: Cai, JY., Cooper, S.B., Li, A. (eds) Theory and Applications of Models of Computation. TAMC 2006. Lecture Notes in Computer Science, vol 3959. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11750321_41
Download citation
DOI: https://doi.org/10.1007/11750321_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34021-8
Online ISBN: 978-3-540-34022-5
eBook Packages: Computer ScienceComputer Science (R0)