Abstract
Despite some successes have been made in face swapping research, face swapping is still not robust and real-time enough. In this paper, a robust and real-time method for face swapping based on face segmentation and CANDIDE-3 is proposed. We implement our method through four steps: face detection, face alignment, modelling and swapping. We test our method on three publicly available datasets and some videos, and results show that our method can effectively improve the robust and real-time performance of face swapping.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Blanz, V., Scherbaum, K., Vetter, T., Seidel, H.P.: Exchanging faces in images. In: Computer Graphics Forum, pp. 669–676 (2004)
Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., Nayar, S.K.: Face swapping: automatically replacing faces in photographs. In: ACM SIGGRAPH, p. 39 (2008)
Yang, F., Wang, J., Shechtman, E., Bourdev, L., Metaxas, D.: Expression flow for 3D-aware face component transfer. Acm Trans. Graph. 30(4), 1–10 (2011)
Lin, Y., Wang, S., Lin, Q., Tang, F.: Face swapping under large pose variations: a 3D model based approach. In: 2012 IEEE International Conference on Multimedia and Expo (ICME), pp. 333–338. IEEE (2012)
Nirkin, Y., Masi, I., Tran, A.T., Hassner, T., Medioni, G.: On face segmentation, face swapping, and face perception. arXiv preprint arXiv:1704.06729 (2017)
De La Hunty, M., Asthana, A., Goecke, R.: Linear facial expression transfer with active appearance models. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 3789–3792. IEEE (2010)
Zhu, J., Van Gool, L., Hoi, S.C.: Unsupervised face alignment by robust nonrigid mapping. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1265–1272. IEEE (2009)
Liu, S., Yang, J., Huang, C., Yang, M.H.: Multi-objective convolutional learning for face labeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3451–3459 (2015)
Qian, K., Wang, B., Chen, H.: Automatic flexible face replacement with no auxiliary data. Comput. Graph. 45, 64–74 (2014)
King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)
Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)
Nguyen, A., Simard-Meilleur, A., Berthiaume, C., Godbout, R., Mottron, L.: Head circumference in canadian male adults: development of a normalized chart. Int. J. Morphol. 30(4), 1474–1480 (2012)
Ahlberg, J.: CANDIDE-3 - an updated parameterised face. Rinsho Byori Jpn. J. Clin. Pathol. 48(3), 385–388 (2001)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Burgos-Artizzu, X.P., Perona, P., Dollar, P.: Robust face landmark estimation under occlusion. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1513–1520 (2013)
Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: IEEE International Conference on Computer Vision Workshops, pp. 397–403 (2014)
Learned-Miller, E., Huang, G.B., Roychowdhury, A., Li, H., Hua, G.: Labeled Faces in the Wild: A Survey. Springer International Publishing, Heidelberg (2016)
Reinhard, E., Adhikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21(5), 34–41 (2001)
Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014)
Bradski, G.: The opencv library. Dr. Dobbs J. Softw. Tools Prof. Program. 25(11), 120–123 (2000)
Jia, X., Yang, H., Chan, K.P., Patras, I.: Structured semi-supervised forest for facial landmarks localization with face mask reasoning. In: BMVC (2014)
Ghiasi, G., Fowlkes, C.: Using segmentation to predict the absence of occluded parts. In: British Machine Vision Conference, pp. 22.1–22.12 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Wang, H., Xie, D., Wei, L. (2018). Robust and Real-Time Face Swapping Based on Face Segmentation and CANDIDE-3. In: Geng, X., Kang, BH. (eds) PRICAI 2018: Trends in Artificial Intelligence. PRICAI 2018. Lecture Notes in Computer Science(), vol 11013. Springer, Cham. https://doi.org/10.1007/978-3-319-97310-4_38
Download citation
DOI: https://doi.org/10.1007/978-3-319-97310-4_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97309-8
Online ISBN: 978-3-319-97310-4
eBook Packages: Computer ScienceComputer Science (R0)