Performance of Image Registration and Its Extensions for Interpolation of Facial Motion
We compare the performance of an intensity based nonparametric image registration algorithm and extensions applied to frame interpolation of mouth images. The mouth exhibits large deformations due to different shapes, additionally some facial features occlude others, e.g. the lips cover the teeth. The closures and disclosures represent a challenging problem, which cannot be solved by the traditional image registration algorithms.
The tested extensions include local regularizer weight adaptation, incorporation of landmarks, self-occlusion handling and penalization of folds, which have all been examined with different weight parameters.
Since the performance of these algorithms and extensions turns out to be superior in case of mouth closures, we provide an algorithm for the automatic selection of deformable template and static reference image for the registration procedure. Subjective tests show that the quality of results for interpolation of mouth images is enhanced by this proposal.
Keywordsframe interpolation facial motion nonparametric image registration variational optical flow numeric optimization
- 2.Liu, K., Weissenfeld, A., Ostermann, J., Luo, X.: Robust AAM building for morphing in an image-based facial animation system. In: 2008 IEEE International Conference on Multimedia and Expo, pp. 933–936 (2008)Google Scholar
- 3.Modersitzki, J.: Numerical Methods for Image Registration. Oxford University Press (2003)Google Scholar
- 4.Modersitzki, J.: FAIR: Flexible Algorithms for Image Registration. SIAM (2009)Google Scholar
- 5.Nocedal, J., Wright, S.J.: Numerical Optimization. Springer (2006)Google Scholar
- 7.Papenberg, N., Olesch, J., Lange, T., Schlag, P., Fischer, B.: Landmark Constrained Non-parametric Image Registration with Isotropic Tolerances. In: Bildverarbeitung für die Medizin 2009, pp. 122–126. Springer, Heidelberg (2009)Google Scholar
- 8.Gay-Bellile, V., Bartoli, A., Sayd, P.: Direct Estimation of Non-Rigid Registrations with Image-Based Self-Occlusion Reasoning. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–6 (2007)Google Scholar
- 10.Bacivarov, I., Ionita, M.C., Corcoran, P.: A combined approach to feature extraction for mouth characterization and tracking. In: Signals and Systems Conference, 208 (ISSC 2008), pp. 156–161. IET Irish (2008)Google Scholar