Skip to main content
SpringerLink
Log in

Dictionary Learning Problem

  • Chapter
  • First Online:
Dictionary Learning Algorithms and Applications

  • 1314 Accesses

Abstract

Dictionary learning can be formulated as an optimization problem in several ways. We present here the basic form, where the representation error is minimized under the constraint of sparsity, and discuss several views and relations with other data analysis and signal processing problems. We study some properties of the DL problem and their implications for the optimization process and explain the two subproblems that are crucial in DL algorithms: sparse coding and dictionary update. In preparation to algorithms analysis and comparisons, we present the main test problems, dealing with representation error and dictionary recovery; we give all details of test procedures, using either artificial data or images. Finally, as an appetizer for the remainder of the book, we illustrate the wide use of DL algorithms in the context of sparse representations for several applications like denoising, inpainting, compression, compressed sensing, classification.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. A. Agarwal, A. Anandkumar, P. Jain, P. Netrapalli, R. Tandon, Learning sparsely used overcomplete dictionaries, in Conference on Learning Theory (2014), pp. 123–137

    Google Scholar 

  2. M. Aharon, M. Elad, A. Bruckstein, K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)

    Google Scholar 

  3. M. Aharon, M. Elad, A. Bruckstein, On the uniqueness of overcomplete dictionaries, a practical way to retrieve them. Linear Algebra Appl. 416(1), 48–67 (2006)

    Google Scholar 

  4. S. Arora, R. Ge, A. Moitra, New algorithms for learning incoherent overcomplete dictionaries, in Conference on Learning Theory (2014), pp. 779–806

    Google Scholar 

  5. O. Bryt, M. Elad, Compression of facial images using the K-SVD algorithm. J. Vis. Commun. Image Represent. 19(4), 270–282 (2008)

    Google Scholar 

  6. E.J. Candès, L. Wakin, An introduction to compressive sampling. IEEE Signal Process. Mag. 25(2), 21–30 (2008)

    Google Scholar 

  7. E.J. Candès, J. Romberg, T. Tao, Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)

    Google Scholar 

  8. H. Chang, M.K. Ng, T. Zeng, Reducing artifacts in JPEG decompression via a learned dictionary. IEEE Trans Signal Process. 62(3), 718–728 (2014)

    Google Scholar 

  9. X. Chen, Z. Du, J. Li, X. Li, H. Zhang, Compressed sensing based on dictionary learning for extracting impulse components. Signal Process. 96, 94–109 (2014)

    Google Scholar 

  10. D.L. Donoho, Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Google Scholar 

  11. D.L. Donoho, J.M. Johnstone, Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3), 425–455 (1994)

    Google Scholar 

  12. M.F. Duarte, M.A. Davenport, D. Takbar, J.N. Laska, T. Sun, K.F. Kelly, R.G. Baraniuk, Single-pixel imaging via compressive sampling. IEEE Signal Process. Mag. 25(2), 83–91 (2008)

    Google Scholar 

  13. J.M. Duarte-Carvajalino, G. Sapiro, Learning to sense sparse signals: simultaneous sensing matrix and sparsifying dictionary optimization. IEEE Trans. Image Process. 18(7), 1395–1408 (2009)

    Google Scholar 

  14. M. Elad, M. Aharon, Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006)

    Google Scholar 

  15. M. Elad, J.L. Starck, P. Querre, D.L. Donoho, Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl. Comput. Harmon. Anal. 19(3), 340–358 (2005)

    Google Scholar 

  16. M.J. Fadili, J.L. Starck, F. Murtagh, Inpainting and zooming using sparse representations. Comput. J. 52(1), 64–79 (2007)

    Google Scholar 

  17. A. Gersho, R.M. Gray, Vector Quantization and Signal Compression (Springer Science & Business Media, Berlin, 1992)

    Google Scholar 

  18. C. Guillemot, O. Le Meur, Image Inpainting. IEEE Signal Process. Mag. 31(1), 127–144 (2014)

    Google Scholar 

  19. O.G. Guleryuz, Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-Part II: adaptive algorithms. IEEE Trans. Image Process. 15(3), 555–571 (2006)

    Google Scholar 

  20. C. Jung, L. Jiao, H. Qi, T. Sun, Image deblocking via sparse representation. Signal Process. Image Commun. 27(6), 663–677 (2012)

    Google Scholar 

  21. S. Lloyd, Least squares quantization in PCM. IEEE Trans. Info. Theory 28(2), 129–137 (1982)

    Google Scholar 

  22. J. Mairal, M. Elad, G. Sapiro, Sparse representation for color image restoration. IEEE Trans. Image Process. 17(1), 53–69 (2008)

    Google Scholar 

  23. K. Marwah, G. Wetzstein, Y. Bando, R. Raskar, Compressive light field photography using overcomplete dictionaries and optimized projections. ACM Trans. Graph. 32(4), 46 (2013)

    Google Scholar 

  24. J.F. Murray, K. Kreutz-Delgado, Sparse image coding using learned overcomplete dictionaries, in International Workshop on Machine Learning for Signal Processing (2004), pp. 579–588

    Google Scholar 

  25. B.A. Olshausen, D.J. Field, Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)

    Google Scholar 

  26. J. Portilla, V. Strela, M.J. Wainwright, E.P. Simoncelli, Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Trans. Image Process. 12(11), 1338–1351 (2003)

    Google Scholar 

  27. S. Ravishankar, Y. Bresler, MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans. Med. Imag. 30(5), 1028–1041 (2011)

    Google Scholar 

  28. R. Rubinstein, A.M. Bruckstein, M. Elad, Dictionaries for sparse representations modeling. Proc. IEEE 98(6), 1045–1057 (2010)

    Google Scholar 

  29. L. Shao, R. Yan, X. Li, Y. Liu, From heuristic optimization to dictionary learning: a review and comprehensive comparison of image denoising algorithms. IEEE Trans. Cybern. 44(7), 1001–1013 (2014)

    Google Scholar 

  30. K. Skretting, K. Engan, Image compression using learned dictionaries by RLS-DLA and compared with K-SVD, in International Conference on Acoustics Speech Signal Processing (ICASSP), Prague (2011), pp. 1517–1520

    Google Scholar 

  31. J.L. Starck, E.J. Candès, D.L. Donoho, The curvelet transform for image denoising. IEEE Trans. Image Process. 11(6), 670–684 (2002)

    Google Scholar 

  32. J. Sun, Q. Qu, J. Wright, Complete dictionary recovery over the sphere, in International Conference on Sampling Theory and Applications (SampTA) (2015), pp. 407–410

    Google Scholar 

  33. I. Tosic, P. Frossard, Dictionary learning. IEEE Signal Process. Mag. 28(2), 27–38 (2011)

    Google Scholar 

  34. D. Vainsencher, S. Mannor, A.M. Bruckstein, The sample complexity of dictionary learning. J. Mach. Learn. Res. 12, 3259–3281 (2011)

    Google Scholar 

  35. Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Google Scholar 

  36. A.G. Weber, The USC-SIPI image database (1997)

    Google Scholar 

  37. M. Xu, X. Jia, M. Pickering, A.J. Plaza, Cloud removal based on sparse representation via multitemporal dictionary learning. IEEE Trans. Geosci. Remote Sens. 54(5), 2998–3006 (2016)

    Google Scholar 

  38. Q. Zhang, B. Li, Dictionary Learning in Visual Computing (Morgan & Claypool, San Rafael, 2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Automatic Control and Systems Engineering, Faculty of Automatic Control and Computers, University Politehnica of Bucharest, Bucharest, Romania

    Bogdan Dumitrescu

  2. Department of Computer Science, Faculty of Mathematics and Computer Science, University of Bucharest, Bucharest, Romania

    Paul Irofti

Authors
  1. Bogdan Dumitrescu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul Irofti
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Dumitrescu, B., Irofti, P. (2018). Dictionary Learning Problem. In: Dictionary Learning Algorithms and Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-78674-2_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-78674-2_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-78673-5

  • Online ISBN: 978-3-319-78674-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation