Abstract
A dictionary should be faithful to the signals it represents, in the sense that the sparse representation error in learning is small, but also must be reliable when recovering sparse representations. A direct way to obtain good recovery guarantees is to modify the objective of the DL optimization such that the resulted dictionary is incoherent, meaning that the atoms are generally far from each other. Alternatively, we can explicitly impose mutual coherence bounds. A related modification of the objective is regularization, in the usual form encountered in least squares problems. Regularization has the additional benefit of making the DL process avoid bottlenecks generated by ill-conditioned dictionaries. We present several algorithms for regularization and promoting incoherence and illustrate their benefits. The K-SVD family can be adapted to such approaches, with good results in the case of regularization. Other methods for obtaining incoherence are based on the gradient of a combined objective including the frame potential or by including a decorrelation step in the standard algorithms. Such a step is based on successive projections on two sets whose properties are shared by the Gram matrix corresponding to the dictionary, which reduce mutual coherence, and rotations of the dictionary, which regain the adequacy to the training signals. We also give a glimpse on the currently most efficient methods that aim at the minimization of the mutual coherence of a frame, regardless of the training signals.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
D. Barchiesi, M.D. Plumbley, Learning incoherent dictionaries for sparse approximation using iterative projections and rotations. IEEE Trans. Signal Process. 61(8), 2055–2065 (2013)
P.G. Casazza, R.G. Lynch, A brief introduction to Hilbert space frame theory and its applications, in Proceedings of Symposia in Applied Mathematics, ed. by K.A. Okoudjou, vol. 73 (American Mathematical Society, Providence, 2016), pp. 1–52
W. Chen, M.R.D. Rodrigues, I.J. Wassell, On the use of unit-norm tight frames to improve the average MSE performance in compressive sensing applications. IEEE Signal Process. Lett. 19(1), 8–11 (2012)
W. Dai, T. Xu, W. Wang, Simultaneous codeword optimization (SimCO) for dictionary update and learning. IEEE Trans. Signal Process. 60(12), 6340–6353 (2012)
P. Davies, N.J. Higham, Numerically stable generation of correlation matrices and their factors. BIT Numer. Math. 40(4), 640–651 (2000)
B. Dumitrescu, Designing incoherent frames with only matrix-vector multiplications. IEEE Signal Process. Lett. 24(9), 1265–1269 (2017)
B. Dumitrescu, P. Irofti, Regularized K-SVD. IEEE Signal Process. Lett. 24(3), 309–313 (2017)
P. Irofti, B. Dumitrescu, Regularized algorithms for dictionary learning, in Proceedings of International Conference Communications, Bucharest (2016), pp. 439–442
T. Lin, S. Liu, H. Zha, Incoherent dictionary learning for sparse representation, in 21st International Conference on Pattern Recognition (ICPR), Tsukuba, November 2012, pp. 1237–1240
J.L. Massey, T. Mittelholzer, Welch’s bound and sequence sets for code-division multiple-access systems, in Sequences II, ed. by R. Capocelli, A. De Santis, U. Vaccaro (Springer, Berlin, 1993), pp. 63–78
D. Needell R. Vershynin, Uniform uncertainty principle and signal recovery via regularized orthogonal matching pursuit. Found. Comput. Math. 9(3), 317–334 (2009)
M. Nejati, S. Samavi, N. Karimi, S.M.R. Soroushmehr, K. Najarian, Boosted dictionary learning for image compression. IEEE Trans. Image Process. 25(10), 4900–4915 (2016)
C. Rusu, N. Gonzalez-Prelcic, Designing incoherent frames through convex techniques for optimized compressed sensing. IEEE Trans. Signal Process. 64(9), 2334–2344 (2016)
M. Sadeghi, M. Babaie-Zadeh, Incoherent unit-norm frame design via an alternating minimization penalty method. IEEE Signal Process. Lett. 24(1), 32–36 (2017)
C.D. Sigg, T. Dikk, J.D. Buhmann, Learning dictionaries with bounded self-coherence. IEEE Signal Process. Lett. 19(19), 861–865 (2012)
J.A. Tropp, I.S. Dhillon, R.W. Heath Jr., T. Strohmer, Designing structured tight frames via an alternating projection method. IEEE Trans. Inf. Theory 51(1), 188–209 (2005)
E.V. Tsiligianni, L.P. Kondi, A.K. Katsaggelos, Construction of incoherent unit norm tight frames with application to compressed sensing. IEEE Trans. Info. Theory 60(4), 2319–2330 (2014)
S. Waldron, Generalized Welch bound equality sequences are tight frames. IEEE Trans. Inf. Theory 49(9), 2307–2309 (2003)
L. Welch, Lower bounds on the maximum cross correlation of signals. IEEE Trans. Inf. Theory 20(3), 397–399 (1974)
M. Yaghoobi, L. Daudet, M.E. Davies, Parametric dictionary design for sparse coding. IEEE Trans. Signal Process. 57(12), 4800–4810 (2009)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Dumitrescu, B., Irofti, P. (2018). Regularization and Incoherence. In: Dictionary Learning Algorithms and Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-78674-2_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-78674-2_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-78673-5
Online ISBN: 978-3-319-78674-2
eBook Packages: EngineeringEngineering (R0)