Advertisement

Learning Overcomplete Representations with a Generalized Gaussian Prior

  • Ling-Zhi Liao
  • Si-Wei Luo
  • Mei Tian
  • Lian-Wei Zhao
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3959)

Abstract

Overcomplete representations have been advocated because they allow a basis to better approximate the underlying statistical density of the data which can lead to representations that better capture the underlying structure in the data. The prior distributions for the coefficients of these models, however, are assumed to be fixed, not adaptive to the data, and hereby inaccurate. Here we describe a method for learning overcomplete representations with a generalized Gaussian prior, which can fit a broader range of statistical distributions by varying the value of the steepness parameter β. Using this distribution in overcomplete representations, empirical results were obtained for the blind source separation of more sources than mixtures, which show that the accuracy of the density estimation is improved.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Lewicki, M.S., Sejnowski, T.J.: Learning Overcomplete Representations. Neural Computation 12(2), 337–365 (2000)CrossRefGoogle Scholar
  2. 2.
    Lewicki, M.S., Olshausen, B.A.: A Probabilistic Framework for the Adaptation and Comparison of Image Codes. J. Opt. Soc. Am. A: Optics, Image Science, and Vision 16(7), 1587–1601 (1999)CrossRefGoogle Scholar
  3. 3.
    Lewicki, M.S., Olshausen, B.A.: Inferring Sparse, Overcomplete Image Codes Using an Effiecient Coding Framework. In: Advances in Neural and Information Processing Systems, vol. 10. Morgan Kaufmann, San Mateo (1998)Google Scholar
  4. 4.
    Olshausen, B.A., Field, D.J.: Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1? Visual Research 37(33), 11–25 (1997)Google Scholar
  5. 5.
    Hyvärinen, A., Oja, E.: Independent Component Analysis: Algorithms and Application. Neural Networks 13(4-5), 411–430 (2000)CrossRefGoogle Scholar
  6. 6.
    Comon, P.: Independent Component Analysis—A new concept? Signal Processing 36(3), 287–314 (1994)CrossRefzbMATHGoogle Scholar
  7. 7.
    : The Generalized Gaussian Mixture Model Using ICA. In: International Workshop on Independent Component Analysis, pp. 239–244. Helsinki (2000)Google Scholar
  8. 8.
    Lee, T.W., Lewicki, M.S., Sejnowski, T.J.: ICA Mixture Models for Unsupervised Classification of Non-Gaussian Sources and Automatic Context Switching in Blind Signal Separation. IEEE Transactions on Pattern Analysis and Machine intelligence 22(10), 1078–1089 (2000)CrossRefGoogle Scholar
  9. 9.
    Amari, S.: Neural Learning in Structured Parameters Spaces. In: Advances in Neural Information Processing Systems, vol. 10. MIT Press, Cambridge (1997)Google Scholar
  10. 10.
    Amari, S.: Natural Gradient Works Efficiently in Learning. Neural Computation 10, 251–276 (1998)CrossRefGoogle Scholar
  11. 11.
    Olshausen, B.A.: Principles of Image Representation in Visual Cortex. In: Chalupa, L.M., Werner, J.S. (eds.) The Visual Neurosciences. MIT Press, Cambridge (2002)Google Scholar
  12. 12.
    Olshausen, B.A., Field, D.J.: Sparse Coding of Sensory Inputs. Current Opinion in Neurobiology 14, 481–487 (2004)CrossRefGoogle Scholar
  13. 13.
    Foldiak, P., Young, M.P.: Sparse Coding in Primate Cortex. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks, 2nd edn. The MIT Press, Cambridge (2002)Google Scholar
  14. 14.
    Olshausen, B.A., Millman, K.J.: Learning Sparse Codes with a Mixture-of-Gaussian Prior. In: Solla, S.A., Leen, T.K., Müller, K.-R. (eds.) Advances in Neural Information Processing Systems, vol. 12. MIT press, Cambridge (2000)Google Scholar
  15. 15.
    Lee, T.W., Lewicki, M.S., Sejnowski, T.J.: ICA Mixture Models for Unsupervised Classification of non-Gaussian Classes and Automatic Context Switching in Blind Signal Separation. IEEE Transactions on Pattern Recognition Machine Intelligence 22(10), 1078–1089 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ling-Zhi Liao
    • 1
  • Si-Wei Luo
    • 1
  • Mei Tian
    • 1
  • Lian-Wei Zhao
    • 1
  1. 1.School of Computer and Information TechnologyBeijing Jiaotong UniversityBeijingChina

Personalised recommendations