Multimedia Tools and Applications

, Volume 56, Issue 3, pp 569–596 | Cite as

Face-off: automatic alteration of facial features

  • Jia-Kai Chou
  • Chuan-Kai YangEmail author
  • Sing-Dong Gong


Pursuing or maintaining beautifulness nowadays has become a trend in modern society, especially among the celebrity community. In some cases, one may choose to adopt drastic procedures to alter his or her facial or body features to achieve the desired beauty, thus the blossom of industry on cosmetic plastic surgeries. In addition, people whose faces got damaged due to accidental burns or wounds may also find these surgeries necessary. However, as performing the related surgeries are still considered intrusive and costly, it is better to “preview” the result before a surgery is actually carried out. As many believe that facial appearance matters most, we have developed a system that allows a user to input a photo and changes the associated individual facial feature in an automatic and user-friendly manner. Overall speaking, our system makes contributions in the following four aspects. First, our system not only offers the previewing functionality, but also allows users to interactively fine-tune the desired results, thus making it a useful companion tool for facial cosmetic surgeries. Second, instead of exchanging the overall look of a face, as being done by some existing approaches, our system offers much finer granularity by allowing each and every facial feature to be changed individually and independently, thus achieving higher face-off flexibility. Third, while existing tools generally entail manual effort to locate or align facial features, our system, through the help of Active Shape Model or ASM for short, characterized by a scheme of automatic feature extraction, eliminates most of the needs of user assistance. Finally, for convenience, we have constructed a database of facial features to facilitate the facial feature alteration process. To justify our claims, we have rendered results and compared them with those from existing approaches to demonstrate the effectiveness of our system. We have also conducted a user study to further confirm the usefulness of such a system.


Poisson image editing Feature extraction Active shape model Facial feature alteration 


  1. 1.
    Adelson EH, Anderson CH, Bergen JR, Burt PJ, Ogden JM (1984) Pyramid method in image processing. RCA Eng 29(6):33–41Google Scholar
  2. 2.
    Agarwala A, Dontcheva M, Agrawala M, Drucker S, Colburn A, Curless B, Salesin D, Cohen M (2004) Interactive digital photomontage. In: SIGGRAPH ’2004, pp 294–302Google Scholar
  3. 3.
    Bertalmio M, Sapiro G, Caselles V, Ballester C (2000) Image inpainting. In: SIGGRAPH ’2000Google Scholar
  4. 4.
    Bitouk D, Kumar N, Dhilon S, Belhumeur P, Nayar SK (2008) Face swapping: automatically replacing faces in photographs. In: SIGGRAPH ’2008Google Scholar
  5. 5.
    Blanz V, Scherbaum K, Vetter T, Seidel H (2004) Exchanging faces in images. In: Eurographics ’2004Google Scholar
  6. 6.
    Blanz V, Vetter T (1999) A morphable model for the synthesis of 3D faces. In: SIGGRAPH ’1999Google Scholar
  7. 7.
    Bornard R, Lecan E, Laborelli L, Chenot J (2002) Missing data correction in still images and image sequences. In: ACM multimedia ’2002, pp 355–361Google Scholar
  8. 8.
    Boykov Y, Jolly MP (2001) Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In: ICCV ’2001Google Scholar
  9. 9.
    Cootes T, Taylor CJ, Cooper DH, Graham J (1995) Active shape models—their training and application. Comput Vis Image Underst 61(1):38–59CrossRefGoogle Scholar
  10. 10.
    Cootes TF, Taylor CJ (2001) Statistical models of appearance for medical image analysis and computer vision. In: Proc. SPIE medical imaging, pp 236–248Google Scholar
  11. 11.
    Efros AA, Leung TK (1999) Texture synthesis by non-parametric sampling. In: ICCV ’99Google Scholar
  12. 12.
    Gu H, Su G, Du C (2003) Feature points extraction from faces. In: Image and vision computing. NZGoogle Scholar
  13. 13.
    Jia J, Sun J, Tang C, Shum H (2006) Drag-and-drop pasting. In: SIGGRAPH ’2006, pp 631–637Google Scholar
  14. 14.
    Kjeldsen R, Kender J (1996) Finding skin in color images. In: FG’ 96 (2nd international conference on automatic face and gesture recognition)Google Scholar
  15. 15.
    Kwatra V, Schodl A, Essa I, Bobick A (2003) Graphcut textures: image and video synthesis using graph cuts. In: SIGGRAPH ’2003Google Scholar
  16. 16.
    Lalonde J, Hoiem D, Efros AA, Rother C, Winn J, Criminisi A (2007) Photo clip art. In: SIGGRAPH ’2007Google Scholar
  17. 17.
    Levin A, Zomet A, Peleg S, Weiss Y (2004) Seamless image stitching in the gradient domain. In: ECCV ’2004, pp 377–389Google Scholar
  18. 18.
    Leyvand T, Cohen-Or D, Dror G, Lischinski D (2006) Digital face beautification. In: SIGGRAPH 2006 technical sketchGoogle Scholar
  19. 19.
    Leyvand T, Cohen-Or D, Dror G, Lischinski D (2008) Data-driven enhancement of facial attractiveness. In: SIGGRAPH ’2008Google Scholar
  20. 20.
    Mortensen EN, Barrett WA (1995) Intelligent scissors for image composition. In: SIGGRAPH ’1995, pp 191–198Google Scholar
  21. 21.
    Oliveira MM, Bowen B, McKenna R, Chang Y (2001) Fast digital image inpainting. In: VIIP (International conference on visualization, imaging and image processing) ’2001Google Scholar
  22. 22.
    Peleg S (1981) Elimination seams from photomosaics. In: Conference on pattern recognition and image processing, pp 426–429Google Scholar
  23. 23.
    Perez P, Gangnet M, Blake A (2003) Poisson image editing. In: SIGGRAPH ’2003, pp 313–318Google Scholar
  24. 24.
    Sun J, Yuan L, Jia J, Shum H (2005) Image completion with structural propagation. In: SIGGRAPH ’2005, pp 861–868Google Scholar
  25. 25.
    Uyttendaele M, Eden A, Szeliski R (2001) Eliminating ghosting and exposure artifacts in image mosaics. In: CVPR ’2001, pp 509–516Google Scholar
  26. 26.
    Wang J, Agrawala M, Cohen M (2007) Soft scissors: an interactive tool for realtime high quality matting. In: SIGGRAPH ’2007Google Scholar
  27. 27.
    Wei L, Levoy M (2000) Fast texture synthesis using tree-structured vector quantization. In: SIGGRAPH ’2000, pp 479–488Google Scholar
  28. 28.
    Yang C, Chiang W (2008) An interactive facial expression generation system. Multimed Tools Appl 40(1):41–60CrossRefGoogle Scholar
  29. 29.
    Yang M, Kriegman DJ, Ahuja N (2002) Detecting faces in images: a survey. IEEE Trans Pattern Anal Mach Intell 24(1):34–58CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Department of Information ManagementNational Taiwan University of Science and TechnologyTaipeiRepublic of China

Personalised recommendations