Schwarps: Locally Projective Image Warps Based on 2D Schwarzian Derivatives

  • Rahat Khan
  • Daniel Pizarro
  • Adrien Bartoli
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8692)


Image warps -or just warps- capture the geometric deformation existing between two images of a deforming surface. The current approach to enforce a warp’s smoothness is to penalize its second order partial derivatives. Because this favors locally affine warps, this fails to capture the local projective component of the image deformation. This may have a negative impact on applications such as image registration and deformable 3D reconstruction. We propose a novel penalty designed to smooth the warp while capturing the deformation’s local projective structure. Our penalty is based on equivalents to the Schwarzian derivatives, which are projective differential invariants exactly preserved by homographies. We propose a methodology to derive a set of Partial Differential Equations with homographies as solutions. We call this system the Schwarzian equations and we explicitly derive them for 2D functions using differential properties of homographies. We name as Schwarp a warp which is estimated by penalizing the residual of Schwarzian equations. Experimental evaluation shows that Schwarps outperform existing warps in modeling and extrapolation power, and lead to far better results in Shape-from-Template and camera calibration from a deformable surface.


Schwarzian Penalizer Bending Energy Projective Differential Invariants Image Warps 


  1. 1.
    Astrom, K.: Fundamental limitations on projective invariants of planar curves. IEEE Transactions on Pattern Analysis and Machine Intelligence 17(1), 77–81 (1995)CrossRefGoogle Scholar
  2. 2.
    Bartoli, A., Gérard, Y., Chadebecq, F., Collins, T.: On template-based reconstruction from a single view: Analytical solutions and proofs of well-posedness for developable, isometric and conformal surfaces. In: Computer Vision and Pattern Recognition (2012)Google Scholar
  3. 3.
    Bartoli, A., Zisserman, A.: Direct estimation of non-rigid registrations. In: British Machine Vision Conference (2004)Google Scholar
  4. 4.
    Bartoli, A., Collins, T.: Template-based isometric deformable 3D reconstruction with sampling-based focal length self-calibration. In: Computer Vision and Pattern Recognition (2013)Google Scholar
  5. 5.
    Bartoli, A., Perriollat, M., Chambon, S.: Generalized thin-plate spline warps. International Journal of Computer Vision 88(1), 85–110 (2010)CrossRefGoogle Scholar
  6. 6.
    Bookstein, F.L.: Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Transaction on Pattern Analysis Machine Intelligence 11(6), 567–585 (1989)CrossRefzbMATHGoogle Scholar
  7. 7.
    Brunet, F., Bartoli, A., Navab, N., Malgouyres, R.: NURBS warps. In: British Machine Vision Conference (2009)Google Scholar
  8. 8.
    Cayley, A.: On the Schwarzian derivatives and the polyhedral functions. Transaction of Cambridge Philosophical Society 13 (1880)Google Scholar
  9. 9.
    Faugeras, O.D., Luong, Q.T., Papadopoulo, T.: The Geometry of Multiple Images: The Laws that Govern the Formation of Multiple Images of a Scene and Some of Their Applications. MIT Press (2001)Google Scholar
  10. 10.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2003) ISBN: 0521623049Google Scholar
  11. 11.
    Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  12. 12.
    Kummer, E.E.: Über die hypergeometrische reihe. Journal Fur Die Reine Und Angewandte Mathematik 1836(15), 39–83 (1836)Google Scholar
  13. 13.
    Lazebnik, S., Ponce, J.: The local projective shape of smooth surfaces and their outlines. International Journal of Computer Vision 63, 65–83 (2005)CrossRefGoogle Scholar
  14. 14.
    Lei, G.: Recognition of planar objects in 3D space from single perspective views using cross ratio. IEEE Transactions on Robotics and Automation 6(4), 432–437 (1990)CrossRefGoogle Scholar
  15. 15.
    Matsumoto, K., Sasaki, T., Yoshida, M.: Recent progress of gauss-schwartz theory and related geometric structures. Memoirs of the Faculty of Science 47(2), 283–381 (1993)zbMATHMathSciNetGoogle Scholar
  16. 16.
    Molzon, R., Mortensen, K.P.: The Schwarzian derivative for maps between manifolds with complex projective connections. Transactions of the American Mathematical Society 348(8), 3015–3036 (1996)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Mundy, J.L., Zisserman, A.: Geometric invariance in computer vision. MIT Press (1992)Google Scholar
  18. 18.
    Oda, T.: On schwarzian derivatives in several variables. Kokyuroku Research Institute for Mathematical Sciences 226, 82–85 (1974) (in Japanese)Google Scholar
  19. 19.
    Osgood, B., Stowe, D.: The Schwarzian derivative and conformal mapping of Riemannian manifolds. Duke Mathematical Journal 67(1), 57–99 (1992)CrossRefzbMATHMathSciNetGoogle Scholar
  20. 20.
    Ovsienko, V.: Lagrange Schwarzian derivative. Moscow University Mechanics Bulletin 44(6), 8–13 (1989)zbMATHGoogle Scholar
  21. 21.
    Ovsienko, V., Tabachnikov, S.: Projective Differential Geometry Old and New: From the Schwarzian Derivative to the Cohomology of Diffeomorphism Groups. Cambridge University Press (2005)Google Scholar
  22. 22.
    Ovsienko, V., Tabachnikov, S.: What is...the Schwarzian derivative? North American Mathematical Society 56, 34–36 (2009)zbMATHMathSciNetGoogle Scholar
  23. 23.
    Pilet, J., Lepetit, V., Fua, P.: Fast non-rigid surface detection, registration and realistic augmentation. International Journal of Computer Vision 76(2), 109–122 (2007)CrossRefGoogle Scholar
  24. 24.
    Rueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L.G., Leach, M.O., Hawkes, D.J.: Nonrigid registration using free-form deformations: Application to breast MR images. IEEE Transactions on Medical Imaging 18, 712–721 (1999)CrossRefGoogle Scholar
  25. 25.
    Salzmann, M., Pilet, J., Ilic, S., Fua, P.: Surface deformation models for nonrigid 3D shape recovery. Transactions on Pattern Analysis and Machine Intelligence 29(8), 1481–1487 (2007)CrossRefGoogle Scholar
  26. 26.
    Schmid, C., Zisserman, A.: The geometry and matching of lines and curves over multiple views. International Journal of Computer Vision 40, 199–233 (2000)CrossRefzbMATHGoogle Scholar
  27. 27.
    Singer, D.: Stable orbits and bifurcation of maps of the interval. SIAM Journal of Applied Mathematics 35(2), 260–267 (1978)CrossRefzbMATHGoogle Scholar
  28. 28.
    Szeliski, R.: Image alignment and stitching: A tutorial. Foundations and Trends in Computer Graphics and Computer Vision 2(1), 1–104 (2006)CrossRefGoogle Scholar
  29. 29.
    Torr, P.: MLESAC: A new robust estimator with application to estimating image geometry. Computer Vision and Image Understanding 78, 138–156 (2000)CrossRefGoogle Scholar
  30. 30.
    Torresani, L., Hertzmann, A., Bregler, C.: Non-rigid structure-from-motion: Estimating shape and motion with hierarchical priors. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(5), 878–892 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Rahat Khan
    • 1
  • Daniel Pizarro
    • 1
  • Adrien Bartoli
    • 1
  1. 1.ISIT, UMR 6284 CNRS-UdAClermont-FerrandFrance

Personalised recommendations