Abstract
Solving an underdetermined inverse problem implies the use of a regularization hypothesis. Among possible regularizers, the so-called sparsity hypothesis, described as a synthesis (generative) model of the signal of interest from a low number of elementary signals taken in a dictionary, is now widely used. In many inverse problems of this kind, it happens that an alternative model, the cosparsity hypothesis (stating that the result of some linear analysis of the signal is sparse), offers advantageous properties over the classical synthesis model. A particular advantage is its ability to intrinsically integrate physical knowledge about the observed phenomenon, which arises naturally in the remote sensing contexts through some underlying partial differential equation. In this chapter, we illustrate on two worked examples (acoustic source localization and brain source imaging) the power of a generic cosparse approach to a wide range of problems governed by physical laws, how it can be adapted to each of these problems in a very versatile fashion, and how it can scale up to large volumes of data typically arising in applications.
Keywords
Srđan Kitić contributed to the results reported in this chapter when he was with Univ Rennes, Inria, CNRS, IRSIA. The chapter was written while he was a postdoc with Technicolor, Rennes.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
The operators for which \({\langle \mathfrak {L} \boldsymbol {\MakeLowercase {\mathit {p}}}_1, \, \boldsymbol {\MakeLowercase {\mathit {p}}}_2 \rangle = \langle \boldsymbol {\MakeLowercase {\mathit {p}}}_1, \, \mathfrak {L} \boldsymbol {\MakeLowercase {\mathit {p}}}_2 \rangle }\) holds. Otherwise, setting the adjoint boundary conditions would be required.
- 2.
The change of notation, in particular from x/c to z for the unknown, is meant to cover both cases in a generic framework.
- 3.
j denotes an iteration index.
- 4.
g ∗(λ) :=sup z g(z) −z T λ.
- 5.
The spatial dimensions remain fixed to ensure solvability of the inverse problem.
- 6.
Note that, in this simplistic setting, a fast computation of Ω −1 c using the Thomas algorithm [77] could be exploited. The reader is reminded that this is not a generally available commodity, which is the main incentive for considering the analysis counterpart.
- 7.
The value of ∥AΨ∥is actually somewhat lower than ∥Ψ∥; it depends on the number of microphones m and their random placement.
- 8.
This metric is more reliable than the corresponding one with respect to the source signal, since small defects in support estimation should not disproportionately affect performance.
- 9.
Vertical axis, k m, the ratio between the number of sources and sensors; horizontal axis, m s, the proportion of the discretized space occupied by sensors
References
L. Albera, A. Ferreol, D. Cosandier-Rimele, I. Merlet, F. Wendling, Brain source localization using a fourth-order deflation scheme. IEEE Trans. Biomed. Eng. 55(2), 490–501 (2008)
L. Albera, S. Kitic, N. Bertin, G. Puy, R. Gribonval, Brain source localization using a physics-driven structured cosparse representation of EEG signals, in 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP), September 21–24 (2014)
N. Antonello, T. van Waterschoot, M. Moonen, P.A. Naylor, Identification of surface acoustic impedances in a reverberant room using the FDTD method, in 2014 14th International Workshop on Acoustic Signal Enhancement (IWAENC), pp. 114–118 (IEEE, New York, 2014)
J. Barker, R. Marxer, E. Vincent, S. Watanabe, The third chime speech separation and recognition challenge: dataset, task and baselines, in 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), Dec (2015), pp. 504–511
A. Beck, M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2(1), 183–202 (2009)
N. Bertin, S. Kitić, R. Gribonval, Joint estimation of sound source location and boundary impedance with physics-driven cosparse regularization, in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, New York, 2016), pp. 6340–6344
C. Bilen, S. Kitić, N. Bertin, R. Gribonval, Sparse acoustic source localization with blind calibration for unknown medium characteristics, in iTwist-2nd international-Traveling Workshop on Interactions Between Sparse Models and Technology (2014)
C. Blandin, A. Ozerov, E. Vincent, Multi-source TDOA estimation in reverberant audio using angular spectra and clustering. Signal Process. 92(8), 1950–1960 (2012)
T. Blumensath, M.E. Davies, Sampling theorems for signals from the union of finite-dimensional linear subspaces. IEEE Trans. Inf. Theory 55(4), 1872–1882 (2009)
P. Bochev, R.B. Lehoucq, On the finite element solution of the pure Neumann problem. SIAM Rev. 47(1), 50–66 (2005)
S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)
J. Campbell, Introduction to Remote Sensing (Guilford Press, New York, 1996)
E.J. Candès, J.K. Romberg, T. Tao, Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)
A. Chambolle, T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imag. Vis. 40(1), 120–145 (2011)
A. Chambolle, T. Pock, An introduction to continuous optimization for imaging. Acta Numer. 25, 161–319 (2016)
V. Chandrasekaran, M.I. Jordan, Computational and statistical tradeoffs via convex relaxation. Proc. Natl. Acad. Sci. 110(13), E1181–E1190 (2013)
G. Chardon, T. Nowakowski, J. De Rosny, L. Daudet, A blind dereverberation method for narrowband source localization. J. Select. Topics Signal Process. 9, 815–824 (2015)
P.L. Combettes, J.-C. Pesquet, Proximal splitting methods in signal processing, in Fixed-Point Algorithms for Inverse Problems in Science and Engineering (Springer, New York, 2011), pp. 185–212
L. Condat, A primal–dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)
G. Dassios, A. Fokas, The definite non-uniqueness results for deterministic eeg and meg data. Inv. Prob. 29(6), 065012 (2013)
A. Deleforge, F. Forbes, R. Horaud, Acoustic space learning for sound-source separation and localization on binaural manifolds. Int. J. Neural Syst. 25(01), 1440003 (2015)
W. Deng, W. Yin, On the global and linear convergence of the generalized alternating direction method of multipliers. J. Sci. Comput. 66(3), 889–916 (2016)
A. Devaney, G. Sherman, Nonuniqueness in inverse source and scattering problems. IEEE Trans. Antennas Propag. 30(5), 1034–1037 (1982)
I. Dokmanić, Listening to Distances and Hearing Shapes. PhD thesis, EPFL (2015)
I. Dokmanić, M. Vetterli, Room helps: acoustic localization with finite elements, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012 (IEEE, New York, 2012), pp. 2617–2620
P.L. Dragotti, M. Vetterli, T. Blu, Sampling moments and reconstructing signals of finite rate of innovation: Shannon meets Strang-fix. IEEE Trans. Signal Process. 55(5), 1741–1757 (2007)
J. Eckstein, D.P. Bertsekas, On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55(1), 293–318 (1992)
M. Elad, P. Milanfar, R. Rubinstein, Analysis versus synthesis in signal priors. Inv. Prob. 23(3), 947 (2007)
Q. Fang, D.A. Boas, Tetrahedral mesh generation from volumetric binary and grayscale images. in 2009 IEEE International Symposium on Biomedical Imaging from Nano to Macro (IEEE, New York, 2009), pp. 1142–1145
D.C.-L. Fong, M. Saunders, LSMR: an iterative algorithm for sparse least-squares problems. SIAM J. Sci. Comput. 33(5), 2950–2971 (2011)
I. Fried, Bounds on the spectral and maximum norms of the finite element stiffness, flexibility and mass matrices. Int. J. Solids Struct. 9(9), 1013–1034 (1973)
A. Friedman, Partial Differential Equations of Parabolic Type (Courier Dover Publications, New York, 2008)
D. Gabay, B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)
S. Gannot, D. Burshtein, E. Weinstein, Signal enhancement using beamforming and nonstationarity with applications to speech. IEEE Trans. Signal Process. 49(8), 1614–1626 (2001)
M.S. Gockenbach, Understanding and Implementing the Finite Element Method (SIAM, Philadelphia, 2006)
T. Goldstein, B. O’Donoghue, S. Setzer, R. Baraniuk, Fast alternating direction optimization methods. SIAM J. Imag. Sci. 7(3), 1588–1623 (2014)
R. Grech, T. Cassar, J. Muscat, K.P. Camilleri, S.G. Fabri, M. Zervakis, P. Xanthopoulos, V. Sakkalis, B. Vanrumste, Review on solving the inverse problem in EEG source analysis. J. NeuroEng. Rehabil. 7, 5–25 (2008)
H. Hallez, B. Vanrumste, R. Grech, J. Muscat, W. De Clercq, A. Vergult, Y. D ’asseler, K.P. Camilleri, S.G. Fabri, S.V. Huffel, I. Lemahieu, Review on solving the forward problem in EEG source analysis. J. Neuroeng. Rehabil. 4(4) (2007). https://doi.org/10.1186/1743-0003-4-46
M. Herman, T. Strohmer et al., General deviants: an analysis of perturbations in compressed sensing. IEEE J. Sel. Top. Sign. Proces. 4(2), 342–349 (2010)
V. Isakov, Inverse Problems for Partial Differential Equations, vol. 127 (Springer Science & Business Media, Berlin, 2006)
B.H. Jansen, V.G. Rit, Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns? Biol. Cybern. 73(4), 357–366 (1995)
R. Jenatton, J.-Y. Audibert, F. Bach, Structured variable selection with sparsity-inducing norms. J. Mach. Learn. Res. 12, 2777–2824 (2011)
C.R. Johnson, Computational and numerical methods for bioelectric field problems. Crit. Rev. Biomed. Eng. 25(1), 1–81 (1997)
U.S. Kamilov, H. Mansour, Learning optimal nonlinearities for iterative thresholding algorithms. IEEE Signal Process. Lett. 23(5), 747–751 (2016)
Y. Kim, P. Nelson, Optimal regularisation for acoustic source reconstruction by inverse methods. J. Sound Vib. 275(3–5), 463–487 (2004)
S. Kitić, N. Bertin, R. Gribonval, Hearing behind walls: localizing sources in the room next door with cosparsity, in 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, New York, 2014), pp. 3087–3091
S. Kitić, L. Albera, N. Bertin, R. Gribonval, Physics-driven inverse problems made tractable with cosparse regularization. IEEE Trans. Signal Process. 64(2), 335–348 (2016)
K. Kowalczyk, M. van Walstijn, Modeling frequency-dependent boundaries as digital impedance filters in FDTD and K-DWM room acoustics simulations. J. Audio Eng. Soc. 56(7/8), 569–583 (2008)
H. Kuttruff, Room Acoustics (CRC Press, Boca Raton, 2016)
J. Le Roux, P.T. Boufounos, K. Kang, J.R. Hershey, Source localization in reverberant environments using sparse optimization, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2013 (IEEE, New York, 2013), pp. 4310–4314
R.J. LeVeque, Finite Difference Methods for Ordinary and Partial Differential Equations: Steady-State and Time-Dependent Problems (SIAM, Philadelphia, 2007)
T.T. Lin, F.J. Herrmann, Compressed wavefield extrapolation. Geophysics 72(5), SM77–SM93 (2007)
D. Malioutov, A sparse signal reconstruction perspective for source localization with sensor arrays. Master’s thesis (2003)
D. Malioutov, M. Çetin, A.S. Willsky, A sparse signal reconstruction perspective for source localization with sensor arrays. IEEE Trans. Signal Process. 53(8), 3010–3022 (2005)
M. Mohr, B. Vanrumste, Comparing iterative solvers for linear systems associated with the finite difference discretisation of the forward problem in electro-encephalographic source analysis. Med. Biol. Eng. Comput. 41, 75–84 (2003)
J.-J. Moreau, Proximité et dualité dans un espace hilbertien. Bull. de la Société mathématique de France, 93, 273–299 (1965)
J.C. Mosher, R.M. Leahy, Source localization using recursively applied and projected (RAP) music. IEEE Trans. Signal Process. 47(2), 332–340 (1999)
W. Munk, P. Worcester, C. Wunsch, Ocean Acoustic Tomography (Cambridge University Press, Cambridge, 2009)
G. Mur, Absorbing boundary conditions for the finite-difference approximation of the time-domain electromagnetic-field equations. IEEE Trans. Electromagn. Comput. EMC-23(4), 377–382 (1981)
J. Murray-Bruce, P.L. Dragotti, Solving physics-driven inverse problems via structured least squares, in 2016 24th European Signal Processing Conference (EUSIPCO) (IEEE, New York, 2016), pp. 331–335
S. Nam, M.E. Davies, M. Elad, R. Gribonval, The cosparse analysis model and algorithms. Appl. Comput. Harmon. Anal. 34(1), 30–56 (2013)
Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, vol. 87 (Springer Science & Business Media, New York, 2013)
R. Oostenvelda, P. Praamstra, The five percent electrode system for high-resolution EEG and ERP measurements. Clin. Neurophysiol. Elsevier, 112(4), 713–719 (2001)
A. Pezeshki, Y. Chi, L.L. Scharf, E.K. Chong, Compressed sensing, sparse inversion, and model mismatch, in Compressed Sensing and Its Applications (Springer, New York, 2015), pp. 75–95
T. Piotrowski, D. Gutierrez, I. Yamada, J. Zygierewicz, Reduced-rank neural activity index for EEG/MEG multi-source localization, in ICASSP’14, IEEE International Conference on Acoustics Speech and Signal Processing, Florence, May 4–9 (2014)
L.C. Potter, E. Ertin, J.T. Parker, M. Cetin, Sparsity and compressed sensing in radar imaging. Proc. IEEE 98(6), 1006–1020 (2010)
J. Provost, F. Lesage, The application of compressed sensing for photo-acoustic tomography. IEEE Trans. Med. Imag. 28(4), 585–594 (2009)
A. Rosenthal, D. Razansky, V. Ntziachristos, Fast semi-analytical model-based acoustic inversion for quantitative optoacoustic tomography. IEEE Trans. Med. Imag. 29(6), 1275–1285 (2010)
Y. Saad, Iterative Methods for Sparse Linear Systems (SIAM, Philadelphia, 2003)
O. Scherzer, M. Grasmair, H. Grossauer, M. Haltmeier, F. Lenzen, Variational Methods in Imaging, vol. 320 (Springer, Berlin, 2009)
P. Schimpf, C. Ramon, J. Haueisen, Dipole models for the EEG and MEG. IEEE Trans. Biomed. Eng. 49(5), 409–418 (2002)
R. Schmidt, Multiple emitter location and signal parameter estimation. IEEE Trans. Antennas Propag. 34(3), 276–280 (1986). Reprint of the original 1979 paper from the RADC Spectrum Estimation Workshop
S. Shalev-Shwartz, N. Srebro, SVM optimization: inverse dependence on training set size, in Proceedings of the 25th international conference on Machine learning (ACM, New York, 2008), pp. 928–935
R. Shefi, M. Teboulle, Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization. SIAM J. Optim. 24(1), 269–297 (2014)
J.-L. Starck, J.M. Fadili, An overview of inverse problem regularization using sparsity, in IEEE ICIP, Cairo, Nov (2009), pp. 1453–1456
G. Strang, Computational Science and Engineering, vol. 791 (Wellesley-Cambridge Press, Wellesley, 2007)
L. Thomas, Using a computer to solve problems in physics. Appl. Digital Comput. 458, 44–45 (1963)
J.A. Tropp, S.J. Wright, Computational methods for sparse solution of linear inverse problems. Proc. IEEE 98(6), 948–958 (2010)
K. Uutela, M. Hamalainen, E. Somersalo, Visualization of magnetoencephalographic data using minimum current estimates. Elsevier NeuroImage 10(2), 173–180 (1999)
J.-M. Valin, F. Michaud, J. Rouat, D. Létourneau, Robust sound source localization using a microphone array on a mobile robot, in Proceedings. 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003. (IROS 2003), vol. 2, pp. 1228–1233 (IEEE, New York, 2003)
F. Vatta, F. Meneghini, F. Esposito, S. Mininel, F. Di Salle, Realistic and spherical head modeling for EEG forward problem solution: a comparative cortex-based analysis. Comput. Intell. Neurosci. 2010, 11pp. (2010)
M. Vetterli, P. Marziliano, T. Blu, Sampling signals with finite rate of innovation. IEEE Trans. Signal Process. 50(6), 1417–1428 (2002)
C. Wolters, H. Köstler, C. Möller, J. Härdtlein, A. Anwander, Numerical approaches for dipole modeling in finite element method based source analysis. Int. Congr. Ser. 1300, 189–192 (2007)
Acknowledgements
This work was supported in part by the European Research Council, PLEASE project (ERC-StG-2011-277906).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Kitić, S., Bensaid, S., Albera, L., Bertin, N., Gribonval, R. (2017). Versatile and Scalable Cosparse Methods for Physics-Driven Inverse Problems. In: Boche, H., Caire, G., Calderbank, R., März, M., Kutyniok, G., Mathar, R. (eds) Compressed Sensing and its Applications. Applied and Numerical Harmonic Analysis. Birkhäuser, Cham. https://doi.org/10.1007/978-3-319-69802-1_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-69802-1_10
Published:
Publisher Name: Birkhäuser, Cham
Print ISBN: 978-3-319-69801-4
Online ISBN: 978-3-319-69802-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)