Skip to main content
SpringerLink
Log in

Reverse-Convex Programming for Sparse Image Codes

  • Conference paper
Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR 2005)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 3757))

  • 1933 Accesses

Abstract

Reverse-convex programming (RCP) concerns global optimization of a specific class of non-convex optimization problems. We show that a recently proposed model for sparse non-negative matrix factorization (NMF) belongs to this class. Based on this result, we design two algorithms for sparse NMF that solve sequences of convex second-order cone programs (SOCP).

We work out some well-defined modifications of NMF that leave the original model invariant from the optimization viewpoint. They considerably generalize the sparse NMF setting to account for uncertainty in sparseness, for supervised learning, and, by dropping the non-negativity constraint, for sparsity-controlled PCA.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Horst, R., Tuy, H.: Global Optimization. Springer, Berlin (1996)

    MATH  Google Scholar 

  2. Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. J. of Mach. Learning Res. 5, 1457–1469 (2004)

    MathSciNet  Google Scholar 

  3. Shen, J., Israël, G.W.: A receptor model using a specific non-negative transformation technique for ambient aerosol. Atmospheric Environment 23(10), 2289–2298 (1989)

    Article  Google Scholar 

  4. Paatero, P., Tapper, U.: Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5, 111–126 (1994)

    Article  Google Scholar 

  5. Xu, W., Liu, X., Gong, Y.: Document clustering based on non-negative matrix factorization. In: SIGIR 2003: Proc. of the 26th Ann. Intl. ACM SIGIR Conf. on Res. and Developm. in Info. Retrieval, pp. 267–273. ACM Press, New York (2003)

    Chapter  Google Scholar 

  6. Hoyer, P.O., Hyvärinen, A.: A multi-layer sparse coding network learns contour coding from natural images. Vision Research 42(12), 1593–1605 (2002)

    Article  Google Scholar 

  7. Smaragdis, P., Brown, J.C.: Non-negative matrix factorization for polyphonic music transcription. In: IEEE Workshop on Appl. of Sign. Proc. to Audio and Acoustics, pp. 177–180 (2003)

    Google Scholar 

  8. Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)

    Article  Google Scholar 

  9. Donoho, D., Stodden, V.: When does non-negative matrix factorization give a correct decomposition into parts? In: Adv. in NIPS, vol. 17 (2004)

    Google Scholar 

  10. Li, S.Z., Hou, X.W., Zhang, H.J., Cheng, Q.S.: Learning spatially localized, parts-based representation. In: Proc. of CVPR (2001)

    Google Scholar 

  11. Wang, Y., Jia, Y., Hu, C., Turk, M.: Fisher non-negative matrix factorization for learning local features. In: Proc. Asian Conf. on Comp. Vision (2004)

    Google Scholar 

  12. Littlestone, N., Warmuth, M.: Relating data compression, learnability, and the Vapnik-Chervonenkis dimension. Tech. Rep., Univ. of Calif. Santa Cruz (1986)

    Google Scholar 

  13. Herbrich, R., Williamson, R.C.: Algorithmic luckiness. J. of Mach. Learning Res. 3, 175–212 (2002)

    Article  MathSciNet  Google Scholar 

  14. Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research 37, 3311–3325 (1997)

    Article  Google Scholar 

  15. Sturm, J.F.: Using SeDuMi 1.02, a Matlab toolbox for optimization over symmetric cones (updated version 1.05). Department of Econometrics, Tilburg University, Tilburg, The Netherlands (2001)

    Google Scholar 

  16. ApS, M. (ed.): The MOSEK optimization tools version 3.2 (Revision 8) User’s manual and reference, MOSEK ApS, Denmark (2005)

    Google Scholar 

  17. Lobo, M.S., Vandenberghe, L., Boyd, S., Lebret, H.: Applications of second-order cone programming. Linear Algebra and its Applications (1998)

    Google Scholar 

  18. Tuy, H.: Convex programs with an additional reverse convex constraint. J. of Optim. Theory and Applic. 52, 463–486 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  19. Horst, R., Pardalos, P.M. (eds.): Handbook of Global Optimization. Kluwer Academic Publishers, Dordrecht (1995)

    MATH  Google Scholar 

  20. d’Aspremont, A., Ghaoui, L.E., Jordan, M.I., Lanckriet, G.R.: A direct formulation for sparse PCA using semidefinite programming. In: Adv. in NIPS (2004)

    Google Scholar 

  21. Zou, H., Hastie, T., Tibshirani, R.: Sparse principal component analysis. J. of Comp. a. Graph. Statistics (to appear)

    Google Scholar 

  22. Chennubholta, C., Jepson, A.: Sparse PCA extracting multi-scale structure from data. In: Proc. of ICCV, pp. 641–647 (2001)

    Google Scholar 

  23. Paatero, P.: Least squares formulation of robust non-negative factor analysis. Chemometrics and Intelligent Laboratory Systems 37 (1997)

    Google Scholar 

  24. CBCL, CBCL face database #1. MIT Center For Biological and Computational Learning (2000), http://cbcl.mit.edu/software-datasets

Download references

Author information

Authors and Affiliations

  1. Computer Vision, Graphics, and Pattern Recognition Group, Department of Mathematics and Computer Science, University of Mannheim, 68131, Mannheim, Germany

    Matthias Heiler & Christoph Schnörr

Authors
  1. Matthias Heiler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christoph Schnörr
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of CISE, University of Florida, Gainesville, FL, USA

    Anand Rangarajan

  2. University of Florida, 32611, Gainesville, FL, USA

    Baba Vemuri

  3. Department of Statistics, University of California, Los Angeles

    Alan L. Yuille

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Heiler, M., Schnörr, C. (2005). Reverse-Convex Programming for Sparse Image Codes. In: Rangarajan, A., Vemuri, B., Yuille, A.L. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2005. Lecture Notes in Computer Science, vol 3757. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11585978_39

Download citation

  • DOI: https://doi.org/10.1007/11585978_39

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-30287-2

  • Online ISBN: 978-3-540-32098-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation