Performance of Image Registration and Its Extensions for Interpolation of Facial Motion

  • Stella Grasshof
  • Jörn Ostermann
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8334)


We compare the performance of an intensity based nonparametric image registration algorithm and extensions applied to frame interpolation of mouth images. The mouth exhibits large deformations due to different shapes, additionally some facial features occlude others, e.g. the lips cover the teeth. The closures and disclosures represent a challenging problem, which cannot be solved by the traditional image registration algorithms.

The tested extensions include local regularizer weight adaptation, incorporation of landmarks, self-occlusion handling and penalization of folds, which have all been examined with different weight parameters.

Since the performance of these algorithms and extensions turns out to be superior in case of mouth closures, we provide an algorithm for the automatic selection of deformable template and static reference image for the registration procedure. Subjective tests show that the quality of results for interpolation of mouth images is enhanced by this proposal.


frame interpolation facial motion nonparametric image registration variational optical flow numeric optimization 


  1. 1.
    Dedeoglu, G., Kanade, T., Baker, S.: The Asymmetry of Image Registration and its Application to Face Tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 807–823 (2007)CrossRefGoogle Scholar
  2. 2.
    Liu, K., Weissenfeld, A., Ostermann, J., Luo, X.: Robust AAM building for morphing in an image-based facial animation system. In: 2008 IEEE International Conference on Multimedia and Expo, pp. 933–936 (2008)Google Scholar
  3. 3.
    Modersitzki, J.: Numerical Methods for Image Registration. Oxford University Press (2003)Google Scholar
  4. 4.
    Modersitzki, J.: FAIR: Flexible Algorithms for Image Registration. SIAM (2009)Google Scholar
  5. 5.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer (2006)Google Scholar
  6. 6.
    Johnson, H.J., Christensen, G.E.: Consistent landmark and intensity-based image registration. IEEE Transactions on Medical Imaging 21, 450–461 (2002)CrossRefGoogle Scholar
  7. 7.
    Papenberg, N., Olesch, J., Lange, T., Schlag, P., Fischer, B.: Landmark Constrained Non-parametric Image Registration with Isotropic Tolerances. In: Bildverarbeitung für die Medizin 2009, pp. 122–126. Springer, Heidelberg (2009)Google Scholar
  8. 8.
    Gay-Bellile, V., Bartoli, A., Sayd, P.: Direct Estimation of Non-Rigid Registrations with Image-Based Self-Occlusion Reasoning. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–6 (2007)Google Scholar
  9. 9.
    Gay-Bellile, V., Bartoli, A., Sayd, P.: Direct Estimation of Nonrigid Registrations with Image-Based Self-Occlusion Reasoning. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 87–104 (2010)CrossRefGoogle Scholar
  10. 10.
    Bacivarov, I., Ionita, M.C., Corcoran, P.: A combined approach to feature extraction for mouth characterization and tracking. In: Signals and Systems Conference, 208 (ISSC 2008), pp. 156–161. IET Irish (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Stella Grasshof
    • 1
  • Jörn Ostermann
    • 1
  1. 1.Institut für Informationsverarbeitung (TNT)Germany

Personalised recommendations