Coarse-to-Fine Minimization of Some Common Nonconvexities

  • Hossein Mobahi
  • John W. FisherIII
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8932)


The continuation method is a popular heuristic in computer vision for nonconvex optimization. The idea is to start from a simplified problem and gradually deform it to the actual problem while tracking the solution. There are many choices for how to map the nonconvex objective to some convex task. One popular principle for such construction is Gaussian smoothing of the objective function. This involves an integration which may be expensive to compute numerically. We argue that often simple tricks at the problem formulation plus some mild approximations can make the resulted task amenable to closed form integral.


Continuation Method Diffusion Equation Nonconvex Optimization Graduated Nonconvexity 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barron, J.T., Malik, J.: Color constancy, intrinsic images, and shape estimation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part IV. LNCS, vol. 7575, pp. 57–70. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  2. 2.
    Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: International Conference on Machine Learning, ICML (2009)Google Scholar
  3. 3.
    Bengio, Y.: Learning Deep Architectures for AI. Now Publishers Inc. (2009)Google Scholar
  4. 4.
    Besl, P.J., McKay, N.D.: A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2) (1992)Google Scholar
  5. 5.
    Black, M.J., Rangarajan, A.: On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. International Journal of Computer Vision 19(1), 57–91 (1996)CrossRefGoogle Scholar
  6. 6.
    Blake, A., Zisserman, A.: Visual Reconstruction. MIT Press (1987)Google Scholar
  7. 7.
    Brox, T.: From pixels to regions: partial differential equations in image analysis. Ph.D. thesis, Faculty of Mathematics and Computer Science, Saarland University, Germany (April 2005)Google Scholar
  8. 8.
    Brox, T., Malik, J.: Large displacement optical flow: Descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 500–513 (2011)CrossRefGoogle Scholar
  9. 9.
    Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1996, pp. 303–312. ACM, New York (1996)CrossRefGoogle Scholar
  10. 10.
    Dabov, F., Katkovnik, E.: Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering. IEEE Transactions on Image Processing 16(8), 2080–2095 (2007)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Dhillon, P.S., Keerthi, S.S., Bellare, K., Chapelle, O., Sundararajan, S.: Deterministic annealing for semi-supervised structured output learning. In: AISTATS 2012, vol. 15 (2012)Google Scholar
  12. 12.
    Erhan, D., Manzagol, P.A., Bengio, Y., Bengio, S., Vincent, P.: The difficulty of training deep architectures and the effect of unsupervised pre-training. In: AISTATS, pp. 153–160 (2009)Google Scholar
  13. 13.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D.A., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  14. 14.
    Gehler, P., Chapelle, O.: Deterministic annealing for multiple-instance learning. In: AISTATS 2007, Microtome, Brookline, MA, USA, pp. 123–130 (March 2007)Google Scholar
  15. 15.
    Goldluecke, B., Strekalovskiy, E., Cremers, D.: Tight convex relaxations for vector-valued labeling. SIAM J. Imaging Sciences 6(3), 1626–1664 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  16. 16.
    Goldstein, T., Osher, S.: The split bregman method for l1-regularized problems. SIAM J. Imaging Sciences 2(2), 323–343 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief networks. Neural Computation 18(7), 1527–1554 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Kim, M., Torre, F.D.: Gaussian processes multiple instance learning pp. 535–542 (2010)Google Scholar
  19. 19.
    Krishnamurthy, V., Levoy, M.: Fitting smooth surfaces to dense polygon meshes. In: SIGGRAPH, pp. 313–324 (1996)Google Scholar
  20. 20.
    MacQueen, J.B.: Some methods for classification and analysis of multivariate observations. In: Cam, L.M.L., Neyman, J. (eds.) Proc. of the fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 281–297. University of California Press (1967)Google Scholar
  21. 21.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online dictionary learning for sparse coding. In: Danyluk, A.P., Bottou, L., Littman, M.L. (eds.) ICML. ACM International Conference Proceeding Series, vol. 382, p. 87. ACM (2009)Google Scholar
  22. 22.
    Mobahi, H., Fisher III, J.W.: On the link between gaussian homotopy continuation and convex envelopes. In: Tai, X.-C., Bae, E., Chan, T.F., Lysaker, M. (eds.) EMMCVPR 2015. LNCS, vol. 8932, pp. 43–56. Springer, Heidelberg (2015)Google Scholar
  23. 23.
    Mobahi, H., Rao, S., Ma, Y.: Data-driven image completion by image patch subspaces. In: Picture Coding Symposium (2009)Google Scholar
  24. 24.
    Mobahi, H., Ma, Y., Zitnick, L.: Seeing through the Blur. In: Proceedings of CVPR 2012 (2012)Google Scholar
  25. 25.
    Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Comm. Pure Appl. Math. 42(5), 577–685 (1989)CrossRefzbMATHMathSciNetGoogle Scholar
  26. 26.
    Nielsen, M.: Graduated non-convexity by smoothness focusing. In: Proceedings of the British Machine Vision Conference, p. 60. BMVA Press (1993)Google Scholar
  27. 27.
    Sindhwani, V., Keerthi, S.S., Chapelle, O.: Deterministic annealing for semi-supervised kernel machines. In: ICML 2006, pp. 841–848. ACM, New York (2006)Google Scholar
  28. 28.
    Strekalovskiy, E., Chambolle, A., Cremers, D.: Convex relaxation of vectorial problems with coupled regularization. SIAM J. Imaging Sciences 7(1), 294–336 (2014)CrossRefzbMATHMathSciNetGoogle Scholar
  29. 29.
    Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2432–2439. IEEE (June 2010)Google Scholar
  30. 30.
    Terzopoulos, D.: The computation of visible-surface representations. IEEE Trans. Pattern Anal. Mach. Intell. 10(4), 417–438 (1988)CrossRefzbMATHGoogle Scholar
  31. 31.
    Turk, G., Levoy, M.: Zippered polygon meshes from range images. In: Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1994, pp. 311–318. ACM, New York (1994)Google Scholar
  32. 32.
    Vural, E., Frossard, P.: Analysis of descent-based image registration. SIAM J. Imaging Sciences 6(4), 2310–2349 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  33. 33.
    Watson, L.T.: Theory of globally convergent probability-one homotopies for nonlinear programming. SIAM Journal on Optimization, 761–780 (2001)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Hossein Mobahi
    • 1
  • John W. FisherIII
    • 1
  1. 1.Computer Science and Artificial Intelligence Lab. (CSAIL)Massachusetts Institute of Technology (MIT)USA

Personalised recommendations