Learning PDEs for Image Restoration via Optimal Control
Partial differential equations (PDEs) have been successfully applied to many computer vision and image processing problems. However, designing PDEs requires high mathematical skills and good insight into the problems. In this paper, we show that the design of PDEs could be made easier by borrowing the learning strategy from machine learning. In our learning-based PDE (L-PDE) framework for image restoration, there are two terms in our PDE model: (i) a regularizer which encodes the prior knowledge of the image model and (ii) a linear combination of differential invariants, which is data-driven and can effectively adapt to different problems and complex conditions. The L-PDE is learnt from some input/output pairs of training samples via an optimal control technique. The effectiveness of our L-PDE framework for image restoration is demonstrated with two exemplary applications: image denoising and inpainting, where the PDEs are obtained easily and the produced results are comparable to or better than those of traditional PDEs, which were elaborately designed.
KeywordsOptimal Control Problem Image Restoration Noisy Image Image Denoising Total Variation Regularization
Unable to display preview. Download preview PDF.
- 1.Gabor, D.: Information theory in electron microscopy. Laboratory Investigation 14, 801–807 (1965)Google Scholar
- 4.Witkin, A.: Scale-space filtering. In: International Joint Conference on Artificial Intelligence, IJCAI (1983)Google Scholar
- 7.Chen, T., Shen, J.: Image processing and analysis: variational, PDE, wavelet, and stochastic methods. SIAM Publisher, Philadelphia (2005)Google Scholar
- 8.Tikhonov, A., Arsenin, V.: Solutions of ill-posed problems. Halsted Press (1977)Google Scholar
- 12.Hinze, M., Pinnau, R., Ulbrich, M., Ulbrich, S.: Optimization with PDE contraints. Springer, Heidelberg (2009)Google Scholar
- 13.Papadakis, N., Corpetti, T., Memin, E.: Dynamically consistent optical flow estimation. In: International Conference on Computer Vision, ICCV (2007)Google Scholar
- 14.Papadakis, N., Memin, E.: Variational optimal control technique for the tracking of deformable objects. In: International Conference on Computer Vision, ICCV (2007)Google Scholar
- 18.Stoer, J., Bulirsch, R.: Introduction to numerical analysis, 2nd edn. Springer, Heidelberg (1998)Google Scholar
- 20.Lin, Z., Zhang, W., Tang, X.: Learning partial differential equations for computer vision. Technical report, Microsoft Research, MSR-TR-2008-189 (2008)Google Scholar
- 21.Lin, Z., Zhang, W., Tang, X.: Designing partial differential equations for image processing by combining differental invariants. Technical report, Microsoft Research, MSR-TR-2009-192 (2009)Google Scholar
- 23.Martin, D.R., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: International Conference on Computer Vision, ICCV (2001)Google Scholar
- 24.Roth, S., Black, M.J.: Fields of experts: a framework for learning image priors. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2005)Google Scholar