Discrete Calculus, Optimisation and Inverse Problems in Imaging
 555 Downloads
Abstract
Inverse problems in science are the process of estimating the causal factors that produced a set of observations. Many image processing tasks can be cast as inverse problems: image restoration: noise reduction, deconvolution; segmentation, tomography, demosaicing, inpaiting, and many others, are examples of such tasks. Typically, inverse problems are illposed, and solving these problems efficiently and effectively is a major, ongoing topic of research. While imaging is often thought of as occurring on regular grids, it is also useful to be able to solve these problems on arbitrary graphs. The combined frameworks of discrete calculus and modern optimisation allow us to formulate and provide solutions to many of these problems in an elegant way. This tutorial article summarizes and illustrates some of the research results of the last decade from this point of view. We provide illustrations and major references.
Keywords
Inverse Problem Image Restoration Proximal Point Algorithm Proximity Operator Convex Functional1 Introduction
Inverse problems are prevalent in science, because science is rooted in observations, and observations are nearly always indirect and certainly never perfect: noise is inevitable, instruments suffer from limited bandwidth, various artifacts and far from faultless acquisition components. Yet there is widespread interest in getting the most out of any observations one can make. Indeed, observations at the frontiers of science are typically those that are the faintest, most blurred, noisy and imperfect.
Typically, inverse problems are illposed [26], meaning that their solution varies widely depending on the input data, in particular with noise. A useful approach for solving inverse problems is to use some sort of prior information on the data, called regularization. To formalize this, we turn to statistics.
2 Statistical Interpretation of Inverse Problems
We want to estimate some statistical parameter \(\theta \) on the basis of some observation vector \(\mathsf {x}\).
2.1 Maximum Likelihood
2.2 Maximum a Posteriori
3 Imaging Formulations
The very brief exposition of the previous section covers the basic principle of many statistical methods, including PCA, LDA, EM, Markov Random Fields, Hidden Markov Models, up to graphcut type methods in imaging [5]. Many details can be found in classical texts on pattern analysis [22].
4 Linear Operators
The classical operators in continuousdomain formulations of the problems we have seen so far are the gradient and its adjoint the divergence. These can be easily discretized using finitedifference schemes [20]. Continuous and discrete versions of wavelet operators can also be considered. In the sequel, we choose to define our operators on arbitrary graphs, in the framework of discrete calculus [25].
4.1 The Incidence Matrix
4.2 The DualConstrained Total Variation Model
Among the interesting regularizations, the Total Variation (TV) [35], or ROF model after the initials of its inventors, promotes sparsity of the gradient. In other words, it corresponds to a piecewiseconstant image prior. This is of interest for various reasons, one of which because it is an acceptable model for texturefree natural images. Simplified versions of the MumfordShah model [30] for image segmentation typically use a TV term instead of the more complex piecewisesmooth prior. In [8], authors introduce TV formulations for image restoration in a MAP framework.

Over flat areas: weak gradient implies a strong \(g_i\), itself implyig a strong \(F_{i,j}\) \(\rightarrow \) weak local variations of u.

Near contours: strong gradient implies a weak \(g_i\) itself implying a weak \(F_{i,j}\) \(\rightarrow \) large local variations of u are allowed.
This model is the dualconstrained total variation (DCTV) [17]. To optimize it, we require algorithms capable of dealing with nondifferentiable convex functionals.
5 Algorithms
Optimization algorithms are numerous but research have mostly focused on differentiable methods: gradient descent, conjugate gradient, etc [4], with the exception of the simplex method for linear programming [21]. However nondifferentiable optimization methods have been available at least since the 1960s. The main tool for nondifferentiable optimizing convex functionals is the proximity operator [1, 14, 29, 34]. We recall here the main points.
5.1 Proximity Operator
5.2 Splitting
This fairly simple method extends wellknown ones such as gradient descent and the proximal point algorithm. It can be improved, for instance replacing the gradient descent scheme with Nesterov’s method [32], which in this case yields an optimal convergence rate [31]. The corresponding method is the BeckTeboule proximal gradient method [2].
5.3 PrimalDual Methods
Many splitting methods exist, involving sums of two or more functions, and are detailed in [14]. In the case of (15), the presence of explicit constraints makes the analysis more difficult. Using convex analysis, and in particular the FenchelRockafeller duality, and if the graph is regular, we can optimize it using the Parallel Proximal Algorithm (PPXA) [13]. In the more interesting case of an irregular graph, a primaldual method is necessary [9]. We actually used the algorithm detailed in [6], which has since been generalized [12, 16].
6 Results
DCTV is a flexible framework. In Fig. 4(a,b,c) we restore a blurry, noisy version of an MRI scan using a local regular graph. This is the same image as in Fig. 1. In Fig. 4(d,e,f) we restore an image using an irregular, nonlocal graph. The fine texture of the brick wall has been restored to a high degree. In Fig. 4(g,h,i) we restore a noisy 3D mesh, with the same framework. Only the graph changes.
7 Discussion
Results presented here are interesting to some degree because we have kept the spatial part of the formulation fully discrete, with at its heart a graph representation for the numerical operators we use. However an important point is our assumption that the distribution of image values is continuous. In practice this is not the case and our approach is a relaxation of the reality, since images are typically discretized to 8 or 16bit values. If we require to keep discretized values throughout the formulation, for instance to deal with labeled images, then the approach proposed here would not work. In this case, MRF formulations could be used [18, 19].
We have also kept the discussion in the convex framework. Many important problems are not convex, for instance blind deblurring, where the degradation kernel must be estimated at the same time as the restoration. There exist methods for dealing with nonconvex image restoraton problems, for instance [11], but dealing with nonconvexity and nondifferentiability together remains a challenge in the general case.
8 Conclusion
In this short overview article, we have introduced inverse problems in imaging and a statistical interpretation: the MAP principle for solving inverse problems such as image restoration using apriori information. We have shown how we can use a graph formulation of numerical operators using discrete calculus to propose a general framework for image restoration. This DCTV framework can be optimized using nondifferentiable convex optimization techniques, in particular the proximity operator. We have illustrated this approach on several examples.
DCTV is by no means the only framework available but it is one of the most flexible, fast and effective. With small changes we can tackle very different problems such as mesh or point cloud regularization. In general, the combination of powerful optimization methods, graph representations of spatial information and fast algorithms is a compelling approach for many applications.
References
 1.Bauschke, H., Combettes, P.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011)CrossRefzbMATHGoogle Scholar
 2.Beck, A., Teboulle, M.: A fast iterative shrinkagethresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
 3.Black, M.J., Sapiro, G., Marimont, D.H., Heeger, D.: Robust anisotropic diffusion. IEEE Trans. Image Process. 7(3), 421–432 (1998)CrossRefGoogle Scholar
 4.Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, New York (2004)CrossRefzbMATHGoogle Scholar
 5.Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
 6.BricenoArias, L.M., Combettes, P.L.: A monotone+ skew splitting model for composite monotone inclusions in duality. SIAM J. Optim. 21(4), 1230–1250 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
 7.Candes, E.J., Romberg, J.K., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
 8.Chambolle, A., Caselles, V., Cremers, D., Novaga, M., Pock, T.: An introduction to total variation for image analysis. Theor. Found. Numer. Methods Sparse Recovery 9(263–340), 227 (2010)MathSciNetzbMATHGoogle Scholar
 9.Chambolle, A., Pock, T.: A firstorder primaldual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
 10.Chan, T.F., Golub, G.H., Mulet, P.: A nonlinear primaldual method for total variation based image restoration. SIAM J. Sci. Comput. 20(6), 1964–1977 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
 11.Chouzenoux, E., Jezierska, A., Pesquet, J.C., Talbot, H.: A majorizeminimize subspace approach for \(\ell _{2}\)\(\ell _{0}\) image regularization. SIAM J. Imaging Sci. 6(1), 563–591 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
 12.Combettes, P., Pesquet, J.: Primaldual splitting algorithm for solving inclusions with mixtures of composite, lipschitzian, and parallelsum type monotone operators. SetValued Variational Anal. 20(2), 307–330 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
 13.Combettes, P.L., Pesquet, J.C.: A proximal decomposition method for solving convex variational inverse problems. Inverse Prob. 24(6), 065014 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
 14.Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing. In: Bauschke, H.H., Burachik, R., Combettes, P.L., Elser, V., Luke, D.R., Wolkowicz, H. (eds.) FixedPoint Algorithms for Inverse Problems in Science and Engineering. Springer, New York (2010)Google Scholar
 15.Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forwardbackward splitting. Multiscale Model. Simul. 4(4), 1168–1200 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
 16.Condat, L.: A primaldual splitting method for convex optimization involving lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
 17.Couprie, C., Grady, L., Najman, L., Pesquet, J.C., Talbot, H.: Dual constrained TVbased regularization on graphs. SIAM J. Imaging Sci. 6(3), 1246–1273 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
 18.Couprie, C., Grady, L., Najman, L., Talbot, H.: Power watersheds: a new image segmentation framework extending graph cuts, random walker and optimal spanning forest. In: Proceedings of ICCV, Kyoto, Japan, 2009, pp. 731–738. IEEE (2009)Google Scholar
 19.Couprie, C., Grady, L., Najman, L., Talbot, H.: Power watersheds: a unifying graphbased optimization framework. IEEE Trans. Pattern Anal. Mach. Intell. 33(7), 1384–1399 (2011)CrossRefGoogle Scholar
 20.Courant, R., Friedrichs, K., Lewy, H.: On the partial difference equations of mathematical physics. IBM J. 11(2), 215–234 (1967)MathSciNetCrossRefzbMATHGoogle Scholar
 21.Dantzig, G.B.: Linear Programming and Extensions, 11th edn. Princeton University Press, Princeton (1998)zbMATHGoogle Scholar
 22.Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley & Sons, New York (2001)zbMATHGoogle Scholar
 23.Gilboa, G., Osher, S.: Nonlocal linear image regularization and supervised segmentation. SIAM J. Multiscale Model. Simul. 6(2), 595–630 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
 24.Grady, L.: Random walks for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1768–1783 (2006)CrossRefGoogle Scholar
 25.Grady, L., Polimeni, J.: Discrete Calculus: Applied Analysis on Graphs for Computational Science. Springer Publishing Company, London (2010)CrossRefzbMATHGoogle Scholar
 26.Hadamard, J.: Sur les problèmes aux dérivées partielles et leur signification physique. Princeton university bulletin 13(49–52), 28 (1902)Google Scholar
 27.Jezierska, A., Chouzenoux, E., Pesquet, J.C., Talbot, H., et al.: A convex approach for image restoration with exact poissongaussian likelihood. IEEE Trans. Image Process. 22(2), 828 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
 28.Kolmogorov, A.: Stationary sequences in hilbert space. In: Linear LeastSquares Estimation, p. 66 (1941)Google Scholar
 29.Moreau, J.J.: Fonctions convexes duales et points proximaux dans un espace hilbertien. CR Acad. Sci. Paris Sér. A Math. 255, 2897–2899 (1962)MathSciNetzbMATHGoogle Scholar
 30.Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42(5), 577–685 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
 31.Nemirovsky, A.S., Yudin, D.B., Dawson, E.R.: Problem Complexity and Method Efficiency in Optimization. John Wiley & Sons Ltd, New York (1982)Google Scholar
 32.Nesterov, Y.: Smooth minimization of nonsmooth functions. Math. Program. 103(1), 127–152 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
 33.Perona, P., Malik, J.: Scalespace and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12(7), 629–639 (1990)CrossRefGoogle Scholar
 34.Rockafellar, R.T.: Convex Analysis. Princeton University Press, New Jersey (1970). Reprinted 1997CrossRefzbMATHGoogle Scholar
 35.Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60(1–4), 259–268 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
 36.Smith, S.W.: The Scientist and Engineer’s Guide to Digital Signal Processing. California Technical Publishing, San Diego (1999)Google Scholar
 37.Twomey, S.: Introduction to the Mathematics of Inversion in Remote Sensing and Indirect Measurements. Elsevier, New York (1977)Google Scholar
 38.Widrow, B., Stearns, S.D.: Adaptive Signal Processing. PrenticeHall Inc, Englewood Cliffs (1985)zbMATHGoogle Scholar
 39.Wiener, N.: Extrapolation, Interpolation, and Smoothing of Stationary Time Series, vol. 2. MIT press, Cambridge (1949)zbMATHGoogle Scholar