Super-Resolution of Facial Images in Video at a Distance

  • Bir Bhanu
  • Ju Han
Part of the Advances in Pattern Recognition book series (ACVPR)


In this chapter, we address the problem of super-resolution of facial images in videos that are acquired at a distance. In particular, we consider (a) a closed-loop approach for super-resolution of frontal faces, (b) super-resolution of frontal faces with facial expressions, and (c) super-resolution of side face images. The details of these technical approaches and experimental results are presented in this chapter.


Video Sequence Facial Region Fiducial Point Free Form Deformation Free Form Deformation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 7.
    Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002) CrossRefGoogle Scholar
  2. 8.
    Basri, R., Jacobs, D.W.: Lambertian reflectance and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 218–233 (2003) CrossRefGoogle Scholar
  3. 16.
    Bhanu, B., Zhou, X.: Face recognition from face profile using dynamic time warping. In: Proceedings of International Conference on Pattern Recognition, vol. 4, pp. 499–502 (2004) Google Scholar
  4. 20.
    Blanc, V., Vetter, T.: A morphable model for the synthesis of 3d faces. In: Computer Graphics Proceedings of SIGGRAPH ’99, pp. 187–194 (1999) Google Scholar
  5. 27.
    Capel, D.P., Zisserman, A.: Super-resolution from multiple views using learnt image models. In: CVPR’01, vol. 2, pp. 627–634 (2001) Google Scholar
  6. 40.
    De Lathauwer, L., De Moor, B., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21(4), 1253–1278 (2000) MathSciNetMATHCrossRefGoogle Scholar
  7. 41.
    Dedeoglu, G., Kanade, T., August, J.: High-zoom video hallucination by exploiting spatio-temporal regularities. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, CVPR’04, vol. 2, pp. 151–158 (2004) Google Scholar
  8. 43.
    Elad, M., Feuer, A.: Restoration of a single super-resolution image from several blurred, noisy and under-sampled measured images. IEEE Trans. Image Process. 6, 1646–1658 (1997) CrossRefGoogle Scholar
  9. 57.
    Hager, G.D., Belhumeur, P.N.: Efficient region tracking with parametric models of geometry and illumination. IEEE Trans. Pattern Anal. Mach. Intell. 20(10), 1025–1039 (1998) CrossRefGoogle Scholar
  10. 60.
    Han, J., Bhanu, B.: Performance prediction for individual recognition by gait. Pattern Recognit. Lett. 26(5), 615–624 (2005) CrossRefGoogle Scholar
  11. 66.
    Hewitt, P.A., Dobberfuhl, D.: The science and art of proportionality. Sci. Scope 27, 30–31 (2004) Google Scholar
  12. 68.
    Horn, B.K.P.: Robot Vision. MIT Press, Cambridge (1986) Google Scholar
  13. 73.
    Huang, X., Paragios, N., Metaxas, D.: Shape registration in implicit spaces using information theory and free form deformations. IEEE Trans. Med. Imaging 28(8), 1303–1318 (2006) Google Scholar
  14. 75.
    Irani, M., Peleg, S.: Motion analysis for image enhancement: resolution, occlusion, and transparency. J. Vis. Commun. Image Represent 4, 324–335 (1993) CrossRefGoogle Scholar
  15. 77.
    Jia, K., Gong, S.: Hallucinating multiple occluded face images of different resolutions. Pattern Recognit. Lett. 27(15), 1768–1775 (2006) CrossRefGoogle Scholar
  16. 103.
    Lin, D., Liu, W., Tang, X.: Layered local prediction network with dynamic learning for face super-resolution. In: Proceedings of IEEE International Conference Image Processing, vol. a, pp. 885–888 (2005) Google Scholar
  17. 109.
    Liu, C., Shum, H., Zhang, C.: A two-step approach to hallucinating faces: global parametric model and local nonparametric model. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, CVPR’01, vol. 1, pp. 192–198 (2001) Google Scholar
  18. 110.
    Liu, W., Lin, D., Tang, X.: Hallucinating faces: tensor patch super-resolution and coupled residue compensation. In: CVPR’05, vol. 2, pp. 478–484 (2005) Google Scholar
  19. 129.
    Pan, G., Han, S., Wu, Z., Wang, Y.: Super-resolution of 3d face. In: The nineth European Conference on Computer Vision, ECCV’06, vol. 3952, pp. 389–401 (2006) Google Scholar
  20. 130.
    Park, J.S., Lee, S.W.: Resolution enhancement of facial image using an error back-projection of example-based learning. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 831–836 (2004) Google Scholar
  21. 131.
    Patti, A.J., Sezan, M.I., Tekalp, A.M.: Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Trans. Image Process. 6, 1064–1076 (1997) CrossRefGoogle Scholar
  22. 134.
    Periaswamy, S., Farid, H.: Elastic registration in the presence of intensity variations. IEEE Trans. Med. Imaging 22(7), 865–874 (2003) CrossRefGoogle Scholar
  23. 142.
    Ramamoorthi, R., Hanrahan, P.: On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object. J. Opt. Soc. Am. A 18(10), 2448–2459 (2001) MathSciNetCrossRefGoogle Scholar
  24. 147.
    Rueckert, D., Sonoda, L., Hayes, C., Hill, D., Leach, M., Hawkes, D.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans. Med. Imaging 8, 712–721 (1999) CrossRefGoogle Scholar
  25. 151.
    Schultz, R.R., Stevenson, R.L.: Extraction of high resolution frames from video sequences. IEEE Trans. Image Process. 5(6), 996–1011 (1996) CrossRefGoogle Scholar
  26. 169.
    Tsai, R.Y., Huang, T.S.: Multiframe Image Restoration and Registration, vol. 1. JAI Press, London (1984) Google Scholar
  27. 178.
    Wang, X., Tang, X.: Hallucinating face by eigentransformation. IEEE Trans. Syst. Man Cybern., Part C 35(3), 425–434 (2005) CrossRefGoogle Scholar
  28. 186.
    Xu, Y., Roy-Chowdhury, A.: Integrating the effects of motion, illumination and structure in video sequences. In: Proceedings of IEEE International Conference on Computer Vision, ICCV (2005) Google Scholar
  29. 196.
    Yu, J., Bhanu, B.: Super-resolution restoration of facial images in video. In: International Conference on Pattern Recognition, ICPR’06, vol. 4, pp. 342–345 (2006) Google Scholar
  30. 197.
    Yu, J., Bhanu, B.: Super-resolution of facial images in video with expression changes. In: IEEE International Conference on Advanced Video and Signal-Based Surveillance, Santa Fe, NM, September 1–3, 2008 Google Scholar
  31. 204.
    Zhao, W., Chellapa, R., Phillips, P.J.: Face recognition: a literature survey. ACM Comput. Surv. 35(4), 399–458 (2003) CrossRefGoogle Scholar
  32. 212.
    Zomet, A., Rav-Acha, A., Peleg, S.: Robust super resolution. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, CVPR, vol. 1, pp. 645–650 (2001) Google Scholar

Copyright information

© Springer-Verlag London Limited 2010

Authors and Affiliations

  1. 1.Bourns College of EngineeringUniversity of CaliforniaRiversideUSA
  2. 2.Lawrence Berkeley National LaboratoryUniversity of CaliforniaBerkeleyUSA

Personalised recommendations