International Journal of Computer Vision

, Volume 127, Issue 6–7, pp 957–971 | Cite as

Learning from Longitudinal Face Demonstration—Where Tractable Deep Modeling Meets Inverse Reinforcement Learning

  • Chi Nhan DuongEmail author
  • Kha Gia Quach
  • Khoa Luu
  • T. Hoang Ngan Le
  • Marios Savvides
  • Tien D. Bui


This paper presents a novel subject-dependent deep aging path (SDAP), which inherits the merits of both generative probabilistic modeling and inverse reinforcement learning to model the facial structures and the longitudinal face aging process of a given subject. The proposed SDAP is optimized using tractable log-likelihood objective functions with convolutional neural networks (CNNs) based deep feature extraction. Instead of applying a fixed aging development path for all input faces and subjects, SDAP is able to provide the most appropriate aging development path for individual subject that optimizes the reward aging formulation. Unlike previous methods that can take only one image as the input, SDAP further allows multiple images as inputs, i.e. all information of a subject at either the same or different ages, to produce the optimal aging path for the given subject. Finally, SDAP allows efficiently synthesizing in-the-wild aging faces. The proposed model is experimented in both tasks of face aging synthesis and cross-age face verification. The experimental results consistently show SDAP achieves the state-of-the-art performance on numerous face aging databases, i.e. FG-NET, MORPH, aging faces in the wild (AGFW), and cross-age celebrity dataset (CACD). Furthermore, we also evaluate the performance of SDAP on large-scale Megaface challenge to demonstrate the advantages of the proposed solution.


Face age progression Deep generative models Face aging Subject-dependent deep aging Tractable graphical probabilistic models 


Supplementary material

11263_2019_1165_MOESM1_ESM.pdf (5.1 mb)
Supplementary material 1 (pdf 5255 KB)

Supplementary material 2 (mp4 119181 KB)


  1. Agustsson. E., Timofte, R., & Van Gool, L. (2017). Anchored regression networks applied to age estimation and super resolution. In IEEE International Conference on Computer Vision (ICCV), (pp. 1652–1661).Google Scholar
  2. Angulu, R., & Tapamo, J. R. (2018). Adewumi AO (2018) Age estimation via face images: a survey. EURASIP Journal on Image and Video Processing, 1, 42.CrossRefGoogle Scholar
  3. Antipov, G., Baccouche, M., & Dugelay, JL. (2017). Face aging with conditional generative adversarial networks. arXiv preprint arXiv:1702.01983
  4. Attia, A., & Dayan, S. (2018). Global overview of imitation learning. arXiv preprint arXiv:1801.06503
  5. Burt, D. M., & Perrett, D. I. (1995). Perception of age in adult caucasian male faces: Computer graphic manipulation of shape and colour information. In Proceedings of the Royal Society of London B: Biological Sciences, 259(1355), (pp. 137–143).Google Scholar
  6. Chang, KY., Chen, CS., & Hung, YP. (2011). Ordinal hyperplanes ranker with cost sensitivities for age estimation. In CVPR, IEEE, (pp. 585–592).Google Scholar
  7. Chen, BC., Chen, CS., & Hsu, WH. (2014). Cross-age reference coding for age-invariant face recognition and retrieval. In ECCV, (pp. 768–783).Google Scholar
  8. Chen, C., Yang, W., Wang, Y., Ricanek, K., & Luu, K. (2011). Facial feature fusion and model selection for age estimation. In IEEE Conference on Automatic Face and Gesture Recognition(FG), (pp. 1–7).Google Scholar
  9. Chen, S., Zhang, C., Dong, M., Le, J., & Rao, M. (2017). Using ranking-cnn for age estimation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  10. Choi, S. E., Lee, Y. J., Lee, S. J., Park, K. R., & Kim, J. (2011). Age estimation using a hierarchical classifier based on global and local facial features. Pattern Recognition, 44(6), 1262–1281.CrossRefzbMATHGoogle Scholar
  11. Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001). Active appearance models. IEEE Transactions on Pattern Analysis & Machine Intelligence, 6, 681–685.CrossRefGoogle Scholar
  12. Duan, Y., Chen, X., Houthooft, R., Schulman, J., & Abbeel, P. (2016). Benchmarking deep reinforcement learning for continuous control. In International Conference on Machine Learning, (pp. 1329–1338).Google Scholar
  13. Duong, CN., Luu, K., Quach, KG., & Bui, TD. (2015). Beyond principal components: Deep Boltzmann machines for face modeling. In CVPR, IEEE, (pp. 4786–4794).Google Scholar
  14. Duong, CN., Luu, K., Quach, KG., Bui, TD. (2016). Longitudinal face modeling via temporal deep restricted boltzmann machines. In CVPR, (pp. 5772–5780).Google Scholar
  15. Duong, CN., Quach, KG., Luu, K., Le, N., & Savvides, M. (2017). Temporal non-volume preserving approach to facial age-progression and age-invariant face recognition. In The IEEE International Conference on Computer Vision (ICCV), (pp. 3755–3763).Google Scholar
  16. Duong, C. N., Quach, K. G., Luu, K., Le, H. B., & Ricanek, K. (2011). Fine tuning age estimation with global and local facial features. In IEEE International Conference on Acoustics (pp. 2032–2035). Speech and Signal Processing: Prague, Czech Republic.Google Scholar
  17. FGNET Aging Database (2004).
  18. Finn, C., Levine, S., & Abbeel, P. (2016). Guided cost learning: Deep inverse optimal control via policy optimization. In International Conference on Machine Learning, (pp. 49–58).Google Scholar
  19. Fu, Y., Guo, G., & Huang, T. S. (2010). Age synthesis and estimation via faces: A survey. IEEE transactions on pattern analysis and machine intelligence, 32(11), 1955–1976.CrossRefGoogle Scholar
  20. Geng, X., Smith-Miles, K., & Zhou, ZH. (2008). Facial age estimation by nonlinear aging pattern subspace. In Proceedings of the 16th ACM international conference on Multimedia, ACM, (pp. 721–724).Google Scholar
  21. Geng, X., Zhou, Z. H., & Smith-Miles, K. (2007). Automatic age estimation based on facial aging patterns. PAMI, 29(12), 2234–2240.CrossRefGoogle Scholar
  22. Guo, G., & Wang, X. (2012). A study on human age estimation under facial expression changes. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, IEEE, (pp. 2547–2553).Google Scholar
  23. Guo, G., & Zhang, C. (2014). A study on cross-population age estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 4257–4263).Google Scholar
  24. Guo, G., Mu, G., Fu, Y., & Huang, TS. (2009). Human age estimation using bio-inspired features. In Computer Vision and Pattern Recognition, 2009. CVPR, IEEE Conference on 2009, (pp. 112–119).Google Scholar
  25. Guo, G., Fu, Y., Dyer, C. R., & Huang, T. S. (2008). Image-based human age estimation by manifold learning and locally adjusted robust regression. IEEE Transactions on Image Processing, 17(7), 1178–1188.MathSciNetCrossRefGoogle Scholar
  26. Han, H., Jain, A. K., Wang, F., Shan, S., & Chen, X. (2018). Heterogeneous face attribute estimation: A deep multi-task learning approach. IEEE transactions on pattern analysis and machine intelligence, 40(11), 2597–2609.CrossRefGoogle Scholar
  27. Huo, Z., Yang, X., Xing, C., Zhou, Y., Hou, P., Lv, J., & Geng, X. (2016). Deep age distribution learning for apparent age estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, (pp. 17–24).Google Scholar
  28. Kemelmacher Shlizerman, I., Suwajanakorn, S., & Seitz, SM. (2014). Illumination-aware age progression. In CVPR, IEEE, (pp. 3334–3341).Google Scholar
  29. Kemelmacher-Shlizerman, I., Seitz, SM., Miller, D., Brossard, E. (2016). The megaface benchmark: 1 million faces for recognition at scale. In CVPR, (pp. 4873–4882).Google Scholar
  30. Kwon, Y. H., & da Vitoria, Lobo N. (1999). Age classification from facial images. CVIU, 74(1), 1–21.Google Scholar
  31. Lanitis, A., Draganova, C., & Christodoulou, C. (2004). Comparing different classifiers for automatic age estimation. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 34(1), 621–628.CrossRefGoogle Scholar
  32. Lanitis, A., Taylor, C. J., & Cootes, T. F. (2002). Toward automatic simulation of aging effects on face images. PAMI, 24(4), 442–455.CrossRefGoogle Scholar
  33. Le, T. H. N., Seshadri, K., Luu, K., & Savvides, M. (2015). Facial aging and asymmetry decomposition based approaches to identification of twins. Pattern Recognition, 48(12), 3843–3856.CrossRefGoogle Scholar
  34. Levine, S., & Abbeel, P. (2014). Learning neural network policies with guided policy search under unknown dynamics. In Advances in Neural Information Processing Systems, (pp. 1071–1079).Google Scholar
  35. Li, K., Xing, J., Su, C., Hu, W., Zhang, Y., & Maybank, S. (2018). Deep cost-sensitive and order-preserving feature learning for cross-population age estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 399–408).Google Scholar
  36. Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., & Song, L. (2017). Sphereface: Deep hypersphere embedding for face recognition. arXiv preprint arXiv:1704.08063.
  37. Lou, Z., Alnajar, F., Alvarez, J. M., Hu, N., & Gevers, T. (2018). Expression-invariant age estimation using structured learning. IEEE transactions on pattern analysis and machine intelligence, 40(2), 365–375.CrossRefGoogle Scholar
  38. Luu, K. (2010). Computer approaches for face aging problems. In The 23th Canadian Conference on Artificial Intelligence (CAI), Ottawa, Canada.Google Scholar
  39. Luu, K., Ricanek Jr, K., Bui, TD., & Suen, CY. (2009a). Age estimation using active appearance models and support vector machine regression. In BTAS’09, IEEE, (pp. 1–5).Google Scholar
  40. Luu, K., Suen, C., Bui, T., & Ricanek, JK. (2009b). Automatic child-face age-progression based on heritability factors of familial faces. In BIdS, IEEE, (pp. 1–6).Google Scholar
  41. Niu, Z., Zhou, M., Wang, L., Gao, X., & Hua, G. (2016). Ordinal regression with multiple output cnn for age estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, (pp. 4920–4928).Google Scholar
  42. Pan, H., Han, H., Shan, S., & Chen, X. (2018). Mean-variance loss for deep age estimation from a face. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 5285–5294).Google Scholar
  43. Patterson, E., Ricanek, K., Albert, M., & Boone, E. (2006). Automatic representation of adult aging in facial images. In Proceedings IASTED International Conference Visualization, Imaging, and Image Processing, (pp. 171–176).Google Scholar
  44. Ricanek Jr, K., & Tesafaye, T. (2006). Morph: A longitudinal image database of normal adult age-progression. In FGR 2006, IEEE, (pp. 341–345).Google Scholar
  45. Rothe, R., Timofte, R., & Gool, L. V. (2016). Deep expectation of real and apparent age from a single image without facial landmarks. International Journal of Computer Vision, 126(2–4), 144–157. Scholar
  46. Rowland, D., Perrett, D., et al. (1995). Manipulating facial appearance through shape and color. Computer Graphics and Applications, IEEE, 15(5), 70–76.CrossRefGoogle Scholar
  47. Shen, W., Guo, Y., Wang, Y., Zhao, K., Wang, B., & Yuille, AL. (2018). Deep regression forests for age estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 2304–2313).Google Scholar
  48. Shu, X., Tang, J., Lai, H., Liu, L., & Yan, S. (2015). Personalized age progression with aging dictionary. In ICCV, (pp. 3970–3978).Google Scholar
  49. Suo, J., Chen, X., Shan, S., Gao, W., & Dai, Q. (2012). A concatenational graph evolution aging model. PAMI, 34(11), 2083–2096.CrossRefGoogle Scholar
  50. Suo, J., Zhu, S. C., Shan, S., & Chen, X. (2010). A compositional and dynamic model for face aging. PAMI, 32(3), 385–401.CrossRefGoogle Scholar
  51. Taylor, GW., & Hinton, GE. (2009). Factored conditional restricted boltzmann machines for modeling motion style. In Proceedings of the 26th Annual International Conference on Machine Learning, ACM, (pp. 1025–1032).Google Scholar
  52. Tsai, M. H., Liao, Y. K., & Lin, I. C. (2014). Human face aging with guided prediction and detail synthesis. Multimedia Tools and Applications, 72(1), 801–824.CrossRefGoogle Scholar
  53. Wang, W., Cui, Z., Yan, Y., Feng, J., Yan, S., Shu, X., & Sebe, N. (2016). Recurrent face aging. In CVPR, (pp. 2378–2386).Google Scholar
  54. Wang, F., Han, H., Shan, S., & Chen, X. (2017). Deep multi-task learning for joint prediction of heterogeneous face attributes. In Automatic Face & Gesture Recognition (FG 2017), 2017 12th IEEE International Conference on, IEEE, (pp. 173–179).Google Scholar
  55. Wang, Z., X Tang, WL., & Gao, S. (2018). Face aging with identity-preserved conditional generative adversarial networks. In CVPR.Google Scholar
  56. Xu, J., Luu, K., Savvides, M., Bui, TD., & Suen, CY. (2011). Investigating age invariant face recognition based on periocular biometrics. In International Joint Conference on Biometrics (IJCB), IEEE.Google Scholar
  57. Xu, F., Luu, K., & Savvides, M. (2015). Spartans: Single-sample periocular-based alignment-robust recognition technique applied to non-frontal scenarios. Trans on Image Processing (TIP), 24, 4780–4795.MathSciNetCrossRefzbMATHGoogle Scholar
  58. Yan, S., Liu, M., & Huang, TS. (2008). Extracting age information from local spatially flexible patches. In Acoustics, Speech and Signal Processing, 2008. ICASSP 2008. IEEE International Conference on, IEEE, (pp. 737–740).Google Scholar
  59. Yang, X., Gao, BB., Xing, C., Huo, ZW., Wei, XS., Zhou, Y., Wu, J., & Geng, X. (2015). Deep label distribution learning for apparent age estimation. In Proceedings of the IEEE International Conference on Computer Vision Workshops, (pp. 102–108).Google Scholar
  60. Yang, H., Huang, D., Wang, Y., Jain, AK. (2018). Learning face age progression: A pyramid architecture of gans. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
  61. Yang, H., Huang, D., Wang, Y., Wang, H., & Tang, Y. (2016). Face aging effect simulation using hidden factor analysis joint sparse representation. TIP, 25(6), 2493–2507.MathSciNetzbMATHGoogle Scholar
  62. Yan, S., Wang, H., Fu, Y., Yan, J., Tang, X., & Huang, T. S. (2009). Synchronized submanifold embedding for person-independent pose estimation and beyond. IEEE Transactions on Image Processing, 18(1), 202–210.MathSciNetCrossRefzbMATHGoogle Scholar
  63. Yi, D., Lei, Z., Li, SZ. (2014). Age estimation by multi-scale convolutional network. In Asian conference on computer vision, Springer, (pp. 144–158).Google Scholar
  64. Zhang, Y., & Yeung, DY. (2010). Multi-task warped Gaussian process for personalized age estimation. In CVPR, IEEE, (pp. 2622–2629).Google Scholar
  65. Zhang, Z., Song, Y., & Qi, H. (2017). Age progression/regression by conditional adversarial autoencoder. In CVPR.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science and Software EngineeringConcordia UniversityMontréalCanada
  2. 2.Computer Science and Computer Engineering DepartmentUniversity of ArkansasFayettevilleUSA
  3. 3.CyLab Biometrics Center and the Department of Electrical and Computer EngineeringCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations