Advertisement

Applied Intelligence

, Volume 49, Issue 11, pp 3894–3908 | Cite as

Learning multi-view deep and shallow features through new discriminative subspace for bi-subject and tri-subject kinship verification

  • Oualid LaiadiEmail author
  • Abdelmalik Ouamane
  • Abdelhamid Benakcha
  • Abdelmalik Taleb-Ahmed
  • Abdenour Hadid
Article

Abstract

This paper presents the combination of deep and shallow features (multi-view features) using the proposed metric learning (SILD+WCCN/LR) approach for kinship verification. Our approach based on an automatic and more efficient two-step learning into deep/shallow information. First, five layers for deep features and five shallow features (i.e. texture and shape), representing more precisely facial features involved in kinship relations (Father-Son, Father-Daughter, Mother-Son, and Mother-Daughter) are used to train the proposed Side-Information based Linear Discriminant Analysis integrating Within Class Covariance Normalization (SILD+WCCN) method. Then, each of the features projected through the discriminative subspace of the proposed SILD+WCCN metric learning method. Finally, a Logistic Regression (LR) method is used to fuse the six scores of the projected features. To show the effectiveness of our SILD+WCNN method, we do some experiments on LFW database. In term of evaluation, the proposed automatic Facial Kinship Verification (FKV) is compared with existing ones to show its effectiveness, using two challenging kinship databases. The experimental results showed the superiority of our FKV against existing ones and reached verification rates of 86.20% and 88.59% for bi-subject matching on the KinFaceW-II and TSKinFace databases, respectively. Verification rates for tri-subject matching of 90.94% and 91.23% on the available TSKinFace database for Father-Mother-Son and Father-Mother-Daughter, respectively.

Keywords

Kinship verification Metric learning SILD+WCCN Bi-subject Tri-subject 

Notes

References

  1. 1.
    Alvergne A, Oda R, Faurie C, Matsumoto-Oda A, Durand V, Raymond M (2009) Cross-cultural perceptions of facial resemblance between kin. J Vis 9(6):23.  https://doi.org/10.1167/9.6.23 CrossRefGoogle Scholar
  2. 2.
    Barkan O, Weill J, Wolf L, Aronowitz H (2013) Fast high dimensional vector multiplication face recognition. In: 2013 IEEE international conference on computer vision, pp 1960–1967.  https://doi.org/10.1109/ICCV.2013.246
  3. 3.
    Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27. https://www.csie.ntu.edu.tw/cjlin/libsvm/ Google Scholar
  4. 4.
    Dal Martello MF, Maloney LT (2006) Where are kin recognition signals in the human face? J Vis 6(12):2.  https://doi.org/10.1167/6.12.2 CrossRefGoogle Scholar
  5. 5.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol 1, pp 886–893.  https://doi.org/10.1109/CVPR.2005.177
  6. 6.
    Davis JV, Kulis B, Jain P, Sra S, Dhillon IS (2007) Information-theoretic metric learning. In: Proceedings of the 24th international conference on machine learning, ICML ’07. ACM, New York, pp 209–216.  https://doi.org/10.1145/1273496.1273523. http://doi.acm.org/10.1145/1273496.1273523
  7. 7.
    DeBruine LM, Smith FG, Jones BC, Roberts SC, Petrie M, Spector TD (2009) Kin recognition signals in adult faces. Vis Res 49 (1):38–43.  https://doi.org/10.1016/j.visres.2008.09.025. http://www.sciencedirect.com/science/article/pii/S0042698908004707 CrossRefGoogle Scholar
  8. 8.
    Dehak N, Kenny PJ, Dehak R, Dumouchel P, Ouellet P (2011) Front-end factor analysis for speaker verification. IEEE Trans Audio Speech Lang Process 19(4):788–798.  https://doi.org/10.1109/TASL.2010.2064307 CrossRefGoogle Scholar
  9. 9.
    Goldberger J, Hinton GE, Roweis ST, Salakhutdinov RR (2005) Neighbourhood components analysis. In: Saul LK, Weiss Y, Bottou L (eds) Advances in neural information processing systems 17. MIT Press, pp 513–520. http://papers.nips.cc/paper/2566-neighbourhood-components-analysis.pdf
  10. 10.
    Guillaumin M, Verbeek J, Schmid C (2009) Is that you? metric learning approaches for face identification. In: 2009 IEEE 12th international conference on computer vision, pp 498–505.  https://doi.org/10.1109/ICCV.2009.5459197
  11. 11.
    Harrell FE Jr (2015) Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. Springer, BerlinCrossRefGoogle Scholar
  12. 12.
    Hastie T, Tibshirani R, Friedman J (2001) The elements of statistical learning. Springer series in statistics. Springer, New YorkzbMATHGoogle Scholar
  13. 13.
    Hu J, Lu J, Tan Y (2018) Sharable and individual multi-view metric learning. IEEE Trans Pattern Anal Mach Intell 40(9):2281–2288.  https://doi.org/10.1109/TPAMI.2017.2749576 CrossRefGoogle Scholar
  14. 14.
    Huang GB, Mattar M, Berg T, Learned-Miller E (2008) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In: Workshop on faces in ’real-life’ images: detection, alignment, and recognition. Erik Learned-Miller and Andras Ferencz and Frédéric Jurie, Marseille, France. https://hal.inria.fr/inria-00321923
  15. 15.
    Iman RL, Conover WJ (1982) A distribution-free approach to inducing rank correlation among input variables. Commun Stat Simul Comput 11(3):311–334.  https://doi.org/10.1080/03610918208812265 CrossRefzbMATHGoogle Scholar
  16. 16.
    Kaminski G, Dridi S, Graff C, Gentaz E (2009) Human ability to detect kinship in strangers’ faces: effects of the degree of relatedness. Proc R Soc Lond B Biol Sci 276(1670):3193–3200.  https://doi.org/10.1098/rspb.2009.0677. http://rspb.royalsocietypublishing.org/content/276/1670/3193 CrossRefGoogle Scholar
  17. 17.
    Kaminski G, Ravary F, Graff C, Gentaz E (2010) Firstborns’ disadvantage in kinship detection. Psychol Sci 21(12):1746–1750.  https://doi.org/10.1177/0956797610388045. PMID: 21051523CrossRefGoogle Scholar
  18. 18.
    Kan M, Xu D, Shan S, Li W, Chen X (2013) Learning prototype hyperplanes for face verification in the wild. IEEE Trans Image Process 22:3310–3316CrossRefGoogle Scholar
  19. 19.
    Kannala J, Rahtu E (2012) Bsif: binarized statistical image features. In: Proceedings of the 21st international conference on pattern recognition (ICPR2012), pp 1363–1366Google Scholar
  20. 20.
    Liu C (2014) Discriminant analysis and similarity measure. Pattern Recogn 47(1):359–367.  https://doi.org/10.1016/j.patcog.2013.06.023. http://www.sciencedirect.com/science/article/pii/S0031320313002756 CrossRefGoogle Scholar
  21. 21.
    Lu J, Hu J, Tan YP (2017) Discriminative deep metric learning for face and kinship verification. IEEE Trans Image Process 26(9):4269–4282.  https://doi.org/10.1109/TIP.2017.2717505 MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Lu J, Zhou X, Tan YP, Shang Y, Zhou J (2014) Neighborhood repulsed metric learning for kinship verification. IEEE Trans Pattern Anal Mach Intell 36(2):331–345.  https://doi.org/10.1109/TPAMI.2013.134 CrossRefGoogle Scholar
  23. 23.
    Ma J, Zhao Y, Ahalt S (2002) Osu svm classifier matlab toolbox (ver 3.00). Pulsed Neural Networks. http://svm.sourceforge.net/docs/3.00/api/
  24. 24.
    Kan M, Shan S, Xu D, Chen X (2011) Side-information based linear discriminant analysis for face recognition. In: Proceedings of the BMVC, pp 125.1–125.0.  https://doi.org/10.5244/C.25.125
  25. 25.
    Nguyen HV, Bai L (2011) Cosine similarity metric learning for face verification. Springer, Berlin, pp 709–720.  https://doi.org/10.1007/978-3-642-19309-5_55 Google Scholar
  26. 26.
    Nosaka R, Ohkawa Y, Fukui K (2012) Feature extraction based on co-occurrence of adjacent local binary patterns. Springer, Berlin, pp 82–91.  https://doi.org/10.1007/978-3-642-25346-1_8 Google Scholar
  27. 27.
    Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987.  https://doi.org/10.1109/TPAMI.2002.1017623 CrossRefzbMATHGoogle Scholar
  28. 28.
    Ojansivu V, Heikkilä J (2008) Blur insensitive texture classification using local phase quantization. Springer, Berlin, pp 236–243.  https://doi.org/10.1007/978-3-540-69905-7_27 Google Scholar
  29. 29.
    Owusu E, Zhan Y, Mao QR (2014) An svm-adaboost facial expression recognition system. Appl Intell 40(3):536–545.  https://doi.org/10.1007/s10489-013-0478-9 CrossRefGoogle Scholar
  30. 30.
    Pang Y, Wang S, Yuan Y (2014) Learning regularized lda by clustering. IEEE Trans Neural Netw Learn Syst 25(12):2191–2201.  https://doi.org/10.1109/TNNLS.2014.2306844 CrossRefGoogle Scholar
  31. 31.
    Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: British machine vision conferenceGoogle Scholar
  32. 32.
    Qin X, Tan X, Chen S (2015) Tri-subject kinship verification: understanding the core of a family. IEEE Trans Multimedia 17(10):1855–1867.  https://doi.org/10.1109/TMM.2015.2461462 CrossRefGoogle Scholar
  33. 33.
    Song HO, Xiang Y, Jegelka S, Savarese S (2016) Deep metric learning via lifted structured feature embedding. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 4004–4012.  https://doi.org/10.1109/CVPR.2016.434
  34. 34.
    Swets DL, Weng JJ (1996) Using discriminant eigenfeatures for image retrieval. IEEE Trans Pattern Anal Mach Intell 18(8):831–836.  https://doi.org/10.1109/34.531802 CrossRefGoogle Scholar
  35. 35.
    Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10:207–244. http://dl.acm.org/citation.cfm?id=1577069.1577078 zbMATHGoogle Scholar
  36. 36.
    Wu P, Hoi SC, Xia H, Zhao P, Wang D, Miao C (2013) Online multimodal deep similarity learning with application to image retrieval. In: Proceedings of the 21st ACM international conference on multimedia, MM’13. ACM, New York, pp 153–162.  https://doi.org/10.1145/2502081.2502112. http://doi.acm.org/10.1145/2502081.2502112
  37. 37.
    Wu X, Boutellaa E, López MB, Feng X, Hadid A (2016) On the usefulness of color for kinship verification from face images. In: 2016 IEEE international workshop on information forensics and security (WIFS), pp 1–6.  https://doi.org/10.1109/WIFS.2016.7823901
  38. 38.
    Xing EP, Ng AY, Jordan MI, Russell S (2002) Distance metric learning, with application to clustering with side-information. In: Proceedings of the 15th international conference on neural information processing systems, NIPS’02. MIT Press, Cambridge, pp 521–528. http://dl.acm.org/citation.cfm?id=2968618.2968683
  39. 39.
    Yan H (2017) Kinship verification using neighborhood repulsed correlation metric learning. Image Vis Comput 60:91–97.  https://doi.org/10.1016/j.imavis.2016.08.009. http://www.sciencedirect.com/science/article/pii/S0262885616301330. Regularization Techniques for High-Dimensional Data AnalysisCrossRefGoogle Scholar
  40. 40.
    Yan H, Lu J, Deng W, Zhou X (2014) Discriminative multimetric learning for kinship verification. IEEE Trans Inf Forensics Secur 9(7):1169–1178.  https://doi.org/10.1109/TIFS.2014.2327757 CrossRefGoogle Scholar
  41. 41.
    Yan H, Lu J, Zhou X (2015) Prototype-based discriminative feature learning for kinship verification. IEEE Trans Cybern 45(11):2535–2545.  https://doi.org/10.1109/TCYB.2014.2376934 CrossRefGoogle Scholar
  42. 42.
    Yu H, Chung CY, Wong KP, Lee HW, Zhang JH (2009) Probabilistic load flow evaluation with hybrid latin hypercube sampling and cholesky decomposition. IEEE Trans Power Syst 24(2):661–667.  https://doi.org/10.1109/TPWRS.2009.2016589 CrossRefGoogle Scholar
  43. 43.
    Zhang T, Fang B, Tang YY, Shang Z, Xu B (2010) Generalized discriminant analysis: A matrix exponential approach. IEEE Trans Syst Man Cybern Part B Cybern 40(1):186–197.  https://doi.org/10.1109/TSMCB.2009.2024759 CrossRefGoogle Scholar
  44. 44.
    Zhao YG, Song Z, Zheng F, Shao L (2018) Learning a multiple kernel similarity metric for kinship verification. Inf Sci 430-431:247–260.  https://doi.org/10.1016/j.ins.2017.11.048. http://www.sciencedirect.com/science/article/pii/S0020025516310416 MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Laboratory of LESIAUniversity of BiskraBiskraAlgeria
  2. 2.IEMN DOAE UMR CNRS 8520 LaboratoryPolytechnic University of Hauts-de-FranceValenciennesFrance
  3. 3.University of BiskraBiskraAlgeria
  4. 4.Laboratory of LGEBUniversity of BiskraBiskraAlgeria
  5. 5.Center for Machine Vision and Signal AnalysisUniversity of OuluOuluFinland

Personalised recommendations