Applied Intelligence

, Volume 49, Issue 11, pp 3894–3908 | Cite as

Learning multi-view deep and shallow features through new discriminative subspace for bi-subject and tri-subject kinship verification

  • Oualid LaiadiEmail author
  • Abdelmalik Ouamane
  • Abdelhamid Benakcha
  • Abdelmalik Taleb-Ahmed
  • Abdenour Hadid


This paper presents the combination of deep and shallow features (multi-view features) using the proposed metric learning (SILD+WCCN/LR) approach for kinship verification. Our approach based on an automatic and more efficient two-step learning into deep/shallow information. First, five layers for deep features and five shallow features (i.e. texture and shape), representing more precisely facial features involved in kinship relations (Father-Son, Father-Daughter, Mother-Son, and Mother-Daughter) are used to train the proposed Side-Information based Linear Discriminant Analysis integrating Within Class Covariance Normalization (SILD+WCCN) method. Then, each of the features projected through the discriminative subspace of the proposed SILD+WCCN metric learning method. Finally, a Logistic Regression (LR) method is used to fuse the six scores of the projected features. To show the effectiveness of our SILD+WCNN method, we do some experiments on LFW database. In term of evaluation, the proposed automatic Facial Kinship Verification (FKV) is compared with existing ones to show its effectiveness, using two challenging kinship databases. The experimental results showed the superiority of our FKV against existing ones and reached verification rates of 86.20% and 88.59% for bi-subject matching on the KinFaceW-II and TSKinFace databases, respectively. Verification rates for tri-subject matching of 90.94% and 91.23% on the available TSKinFace database for Father-Mother-Son and Father-Mother-Daughter, respectively.


Kinship verification Metric learning SILD+WCCN Bi-subject Tri-subject 



  1. 1.
    Alvergne A, Oda R, Faurie C, Matsumoto-Oda A, Durand V, Raymond M (2009) Cross-cultural perceptions of facial resemblance between kin. J Vis 9(6):23. CrossRefGoogle Scholar
  2. 2.
    Barkan O, Weill J, Wolf L, Aronowitz H (2013) Fast high dimensional vector multiplication face recognition. In: 2013 IEEE international conference on computer vision, pp 1960–1967.
  3. 3.
    Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27. Google Scholar
  4. 4.
    Dal Martello MF, Maloney LT (2006) Where are kin recognition signals in the human face? J Vis 6(12):2. CrossRefGoogle Scholar
  5. 5.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol 1, pp 886–893.
  6. 6.
    Davis JV, Kulis B, Jain P, Sra S, Dhillon IS (2007) Information-theoretic metric learning. In: Proceedings of the 24th international conference on machine learning, ICML ’07. ACM, New York, pp 209–216.
  7. 7.
    DeBruine LM, Smith FG, Jones BC, Roberts SC, Petrie M, Spector TD (2009) Kin recognition signals in adult faces. Vis Res 49 (1):38–43. CrossRefGoogle Scholar
  8. 8.
    Dehak N, Kenny PJ, Dehak R, Dumouchel P, Ouellet P (2011) Front-end factor analysis for speaker verification. IEEE Trans Audio Speech Lang Process 19(4):788–798. CrossRefGoogle Scholar
  9. 9.
    Goldberger J, Hinton GE, Roweis ST, Salakhutdinov RR (2005) Neighbourhood components analysis. In: Saul LK, Weiss Y, Bottou L (eds) Advances in neural information processing systems 17. MIT Press, pp 513–520.
  10. 10.
    Guillaumin M, Verbeek J, Schmid C (2009) Is that you? metric learning approaches for face identification. In: 2009 IEEE 12th international conference on computer vision, pp 498–505.
  11. 11.
    Harrell FE Jr (2015) Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. Springer, BerlinCrossRefGoogle Scholar
  12. 12.
    Hastie T, Tibshirani R, Friedman J (2001) The elements of statistical learning. Springer series in statistics. Springer, New YorkzbMATHGoogle Scholar
  13. 13.
    Hu J, Lu J, Tan Y (2018) Sharable and individual multi-view metric learning. IEEE Trans Pattern Anal Mach Intell 40(9):2281–2288. CrossRefGoogle Scholar
  14. 14.
    Huang GB, Mattar M, Berg T, Learned-Miller E (2008) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In: Workshop on faces in ’real-life’ images: detection, alignment, and recognition. Erik Learned-Miller and Andras Ferencz and Frédéric Jurie, Marseille, France.
  15. 15.
    Iman RL, Conover WJ (1982) A distribution-free approach to inducing rank correlation among input variables. Commun Stat Simul Comput 11(3):311–334. CrossRefzbMATHGoogle Scholar
  16. 16.
    Kaminski G, Dridi S, Graff C, Gentaz E (2009) Human ability to detect kinship in strangers’ faces: effects of the degree of relatedness. Proc R Soc Lond B Biol Sci 276(1670):3193–3200. CrossRefGoogle Scholar
  17. 17.
    Kaminski G, Ravary F, Graff C, Gentaz E (2010) Firstborns’ disadvantage in kinship detection. Psychol Sci 21(12):1746–1750. PMID: 21051523CrossRefGoogle Scholar
  18. 18.
    Kan M, Xu D, Shan S, Li W, Chen X (2013) Learning prototype hyperplanes for face verification in the wild. IEEE Trans Image Process 22:3310–3316CrossRefGoogle Scholar
  19. 19.
    Kannala J, Rahtu E (2012) Bsif: binarized statistical image features. In: Proceedings of the 21st international conference on pattern recognition (ICPR2012), pp 1363–1366Google Scholar
  20. 20.
    Liu C (2014) Discriminant analysis and similarity measure. Pattern Recogn 47(1):359–367. CrossRefGoogle Scholar
  21. 21.
    Lu J, Hu J, Tan YP (2017) Discriminative deep metric learning for face and kinship verification. IEEE Trans Image Process 26(9):4269–4282. MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Lu J, Zhou X, Tan YP, Shang Y, Zhou J (2014) Neighborhood repulsed metric learning for kinship verification. IEEE Trans Pattern Anal Mach Intell 36(2):331–345. CrossRefGoogle Scholar
  23. 23.
    Ma J, Zhao Y, Ahalt S (2002) Osu svm classifier matlab toolbox (ver 3.00). Pulsed Neural Networks.
  24. 24.
    Kan M, Shan S, Xu D, Chen X (2011) Side-information based linear discriminant analysis for face recognition. In: Proceedings of the BMVC, pp 125.1–125.0.
  25. 25.
    Nguyen HV, Bai L (2011) Cosine similarity metric learning for face verification. Springer, Berlin, pp 709–720. Google Scholar
  26. 26.
    Nosaka R, Ohkawa Y, Fukui K (2012) Feature extraction based on co-occurrence of adjacent local binary patterns. Springer, Berlin, pp 82–91. Google Scholar
  27. 27.
    Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987. CrossRefzbMATHGoogle Scholar
  28. 28.
    Ojansivu V, Heikkilä J (2008) Blur insensitive texture classification using local phase quantization. Springer, Berlin, pp 236–243. Google Scholar
  29. 29.
    Owusu E, Zhan Y, Mao QR (2014) An svm-adaboost facial expression recognition system. Appl Intell 40(3):536–545. CrossRefGoogle Scholar
  30. 30.
    Pang Y, Wang S, Yuan Y (2014) Learning regularized lda by clustering. IEEE Trans Neural Netw Learn Syst 25(12):2191–2201. CrossRefGoogle Scholar
  31. 31.
    Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: British machine vision conferenceGoogle Scholar
  32. 32.
    Qin X, Tan X, Chen S (2015) Tri-subject kinship verification: understanding the core of a family. IEEE Trans Multimedia 17(10):1855–1867. CrossRefGoogle Scholar
  33. 33.
    Song HO, Xiang Y, Jegelka S, Savarese S (2016) Deep metric learning via lifted structured feature embedding. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 4004–4012.
  34. 34.
    Swets DL, Weng JJ (1996) Using discriminant eigenfeatures for image retrieval. IEEE Trans Pattern Anal Mach Intell 18(8):831–836. CrossRefGoogle Scholar
  35. 35.
    Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10:207–244. zbMATHGoogle Scholar
  36. 36.
    Wu P, Hoi SC, Xia H, Zhao P, Wang D, Miao C (2013) Online multimodal deep similarity learning with application to image retrieval. In: Proceedings of the 21st ACM international conference on multimedia, MM’13. ACM, New York, pp 153–162.
  37. 37.
    Wu X, Boutellaa E, López MB, Feng X, Hadid A (2016) On the usefulness of color for kinship verification from face images. In: 2016 IEEE international workshop on information forensics and security (WIFS), pp 1–6.
  38. 38.
    Xing EP, Ng AY, Jordan MI, Russell S (2002) Distance metric learning, with application to clustering with side-information. In: Proceedings of the 15th international conference on neural information processing systems, NIPS’02. MIT Press, Cambridge, pp 521–528.
  39. 39.
    Yan H (2017) Kinship verification using neighborhood repulsed correlation metric learning. Image Vis Comput 60:91–97. Regularization Techniques for High-Dimensional Data AnalysisCrossRefGoogle Scholar
  40. 40.
    Yan H, Lu J, Deng W, Zhou X (2014) Discriminative multimetric learning for kinship verification. IEEE Trans Inf Forensics Secur 9(7):1169–1178. CrossRefGoogle Scholar
  41. 41.
    Yan H, Lu J, Zhou X (2015) Prototype-based discriminative feature learning for kinship verification. IEEE Trans Cybern 45(11):2535–2545. CrossRefGoogle Scholar
  42. 42.
    Yu H, Chung CY, Wong KP, Lee HW, Zhang JH (2009) Probabilistic load flow evaluation with hybrid latin hypercube sampling and cholesky decomposition. IEEE Trans Power Syst 24(2):661–667. CrossRefGoogle Scholar
  43. 43.
    Zhang T, Fang B, Tang YY, Shang Z, Xu B (2010) Generalized discriminant analysis: A matrix exponential approach. IEEE Trans Syst Man Cybern Part B Cybern 40(1):186–197. CrossRefGoogle Scholar
  44. 44.
    Zhao YG, Song Z, Zheng F, Shao L (2018) Learning a multiple kernel similarity metric for kinship verification. Inf Sci 430-431:247–260. MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Laboratory of LESIAUniversity of BiskraBiskraAlgeria
  2. 2.IEMN DOAE UMR CNRS 8520 LaboratoryPolytechnic University of Hauts-de-FranceValenciennesFrance
  3. 3.University of BiskraBiskraAlgeria
  4. 4.Laboratory of LGEBUniversity of BiskraBiskraAlgeria
  5. 5.Center for Machine Vision and Signal AnalysisUniversity of OuluOuluFinland

Personalised recommendations