Multimedia Tools and Applications

, Volume 78, Issue 22, pp 31557–31580 | Cite as

Cancelable fusion-based face recognition

  • Essam AbdellatefEmail author
  • Nabil A. Ismail
  • Salah Eldin S. E. Abd Elrahman
  • Khalid N. Ismail
  • Mohamed Rihan
  • Fathi E. Abd El-Samie


Biometric recognition refers to the automated process of recognizing individuals using their biometric patterns. Recent advancements in deep learning and computer vision indicate that generic descriptors which are extracted using convolutional neural networks (CNNs) could represent complex image characteristics. This paper presents a number of cancelable fusion-based face recognition (FR) methods; region-based, multi-biometric and hybrid-features. The former included methods incorporate the use of CNNs to extract deep features (DFs). A fusion network combines the DFs to obtain a discriminative facial descriptor. Cancelabilitiy is provided using bioconvolving as an encryption method. In the region-based method, the DFs are extracted from different face regions. The multi-biometric method uses different biometric traits to train multiple CNNs. The hybrid-features method merges the merits of deep-learned features and hand-crafted features to obtain a more representative output. Also, an efficient CNN model is proposed. Experimental results on various datasets prove that; (a) the proposed CNN model achieves remarkable results compared to other state-of-the-art CNNs, (b) region-based method is superior to multi-biometric and hybrid-features methods and (c) the utilization of bio-convolving method increases the system security with a slight degradation in the recognition accuracy.


Deep features, Fusion network, and cancelable biometrics 


Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.


  1. 1.
    Agrawal A, Mittal N (2019) Using CNN for facial expression recognition: a study of the effects of kernel size and number of filters on accuracy. Visual Comput J: 1–8Google Scholar
  2. 2.
    Ali H, Jahangir U, Yousuf B, Noor A (2017) Human action recognition using SIFT and HOG method. International Conference on Information and Communication Technologies (ICICT): 6–10Google Scholar
  3. 3.
    AMI ear database (2018) Accessed November
  4. 4.
    Beveridge J, Phillips J, Bolme D, Draper B, Givens G, Lui Y, Teli M, Zhang H, Scruggs W, Bowyer K (2013) The challenge of face recognition from digital point-and-shoot cameras. IEEE Int Conf Biometrics, Theory, Appl Syst: 1–8Google Scholar
  5. 5.
    Canuto A, Pintro F, Xavier-Junior J (2013) Investigation fusion approaches in multi-biometric cancellable recognition. ESA 40:1971–1980Google Scholar
  6. 6.
    CASIA Fingerprint Image Database Version 5.0 (2018) Accessed November
  7. 7.
    CASIA-IrisV3 Database (2018) Accessed May
  8. 8.
    Cheng C, Wang X, Li X (2017) UAV image matching based on surf feature and harris corner algorithm. International Conference on Smart and Sustainable City (ICSSC): 1–6Google Scholar
  9. 9.
    Chowdhury A, Lin T, Maji S, Learned-Miller E (2016) One-tomany face recognition with bilinear cnns. WACV: 1–9Google Scholar
  10. 10.
    COEP Palm Print Database (2018) Accessed December
  11. 11.
    Deng J, Guo J, Zafeiriou S (2018) Arcface: additive angular margin loss for deep face recognition. arXiv preprint arXiv:1801.07698Google Scholar
  12. 12.
    Ding C, Tao D (2017) Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE Trans Pattern Anal Mach IntellGoogle Scholar
  13. 13.
    Gomez-Barrero M, Rathgeb C (2014) Protected facial biometric templates based on local gabor patterns and adaptive bloom filters. ICPR: 4483–4488Google Scholar
  14. 14.
    Hai H, Hao Z, Xu Y, Lu Z, Lu Q, Ai-Yun Z (2019) Faster R-CNN for marine organisms detection and recognition using data augmentation. Neurocomputing: 1–13Google Scholar
  15. 15.
    Hailong H, Yantao L, Zhangqian Z, Gang Z (2019) CNNAuth: continuous authentication via two-stream convolutional neural networks. IEEE NASGoogle Scholar
  16. 16.
    Hasnat A, Bohn’e J, Milgram J, Gentric S, Chen L (2017) DeepVisage: making face recognition simple yet with powerful generalization skills. CVPR: 1–12Google Scholar
  17. 17.
    He X, Yan S, Hu Y, Niyogi P, Zhang J (2005) Face recognition using laplacian faces. IEEE Trans Pattern Anal Mach Intell 3:328–340Google Scholar
  18. 18.
    Huang G, Ramesh M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report. University of Massachusetts, Amherst, pp 07–49Google Scholar
  19. 19.
    Ioffe S, Szegedy CH (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv: 1502.03167v3 [cs.LG]: 1–11Google Scholar
  20. 20.
    Junaid B, Maheen B, Kafil U, Waheed N, Varsha D, Abdul S (2019) Facial expression recognition and analysis of interclass false positives using CNN. Future of Information and Communication Conference: 46–54Google Scholar
  21. 21.
    Kan M, Shan S, Chen X (2016) Multi-view deep network for cross-view classification. CVPR: 4847–4855Google Scholar
  22. 22.
    Kelkboom E, Zhou X (2009) Multi-algorithm fusion with template protection. BTAS:1–7Google Scholar
  23. 23.
    Lin Y, Xun L (2018) A cancelable multi-biometric template generation algorithm based on bloom filter. International Conference on Algorithms and Architectures for Parallel Processing: 547–559Google Scholar
  24. 24.
    Liu J, Deng Y, Bai T, Wei Z, Huang C (2015) Targeting ultimate accuracy: face recognition via deep embedding. arXiv preprint arXiv:1506.07310Google Scholar
  25. 25.
    Liu W, Jia Y, Sermanet P, Rabinovich A (2015) Going deeper with convolutions. IEEE conference on computer vision and pattern recognition: 1–9Google Scholar
  26. 26.
    Liu J, Deng Y, Huang C (2015) Targeting ultimate accuracy: face recognition via deep embedding. arXiv:1506.07310Google Scholar
  27. 27.
    Marco C, Oscar D, Cayetano G, Mario H (2007) ENCARA2: real-time detection of multiple faces at different resolutions in video streams. J Vis Commun Image Represent 18(2):130–140CrossRefGoogle Scholar
  28. 28.
    Masi I, Rawls S, Medioni G, Natarajan P (2016) Pose-aware face recognition in the wild. CVPR: 4838–4846Google Scholar
  29. 29.
    Nagar A, Nandakumar K, Jain A (2012) Multibiometric cryptosystems based on feature-level fusion. IEEE TIFS 7(1):255–268Google Scholar
  30. 30.
    Othman A, Ross A (2013) On mixing fingerprints. IEEE TIFS 8(1):260–267Google Scholar
  31. 31.
    Patel V, Ratha N, Chellappa R (2015) Cancelable biometrics [a review]. IEEE Signal Process Mag, 54–65CrossRefGoogle Scholar
  32. 32.
    Paul P, Gavrilova M (2012) Multimodal cancelable biometrics. ICCI*CC: 43–49Google Scholar
  33. 33.
    Pei S, Chen M, Yu Y, Tang S, Zhong C (2017) Compact LBP and WLBP descriptor with magnitude and direction difference for face recognition. IEEE International Conference on Image Processing (ICIP): 1067–1071Google Scholar
  34. 34.
    Polash P, Gavrilova M, Klimenko S (2014) Situation awareness of cancelable biometric system. Visual Comput J 30:1059–1067CrossRefGoogle Scholar
  35. 35.
    Priya T, Sarika J, Durgesh K (2018) Person-dependent face recognition using histogram of oriented gradients (HOG) and convolution neural network (CNN). International Conference on Advanced Computing Networking and Informatics: 35–40Google Scholar
  36. 36.
    Rajeev R, Vishal M, Rama C (2019) HyperFace: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans Pattern Anal Mach Intell 41(1):121–135CrossRefGoogle Scholar
  37. 37.
    Ranjan R, Sank S, Castillo C, Chellappa R (2017) An all-in-one convolutional neural network for face analysis. FG IEEE: 17–24Google Scholar
  38. 38.
    Ratha N, Connell J, Bolle R (2001) Enhancing security and privacy in biometrics-based authentication systems. IBM SJ 40:614–634CrossRefGoogle Scholar
  39. 39.
    Ratha N, Chikkerur S, Connell J, Bolle R (2007) Generating cancelable fingerprint templates. IEEE Trans Pattern Anal Mach Intell 29:561–572CrossRefGoogle Scholar
  40. 40.
    Rathgeb C, Uhl A (2011) A survey on biometric cryptosystems and cancelable biometrics. EURASIP JIS 3(3)Google Scholar
  41. 41.
    Rathgeb C, Breitinger F, Busch C (2013) Alignment-free cancelable iris biometric templates based on adaptive bloom filters. ICB: 1–8Google Scholar
  42. 42.
    Ross A, Jain A (2003) Information fusion in biometrics. PRL 24(13):2115–2125CrossRefGoogle Scholar
  43. 43.
    Ross A, Nandakumar K, Jain A (2006) Handbook of multi-biometrics, Springer-VerlagGoogle Scholar
  44. 44.
    Schroff F, Kalenichenko D, Philbin J (2015) Facenet: a unified embedding for face recognition and clustering. CVPR:815–823Google Scholar
  45. 45.
    Sun Y, Liang D, Wang X, Tang X (2015) DeepID3: Face recognition with very deep neural networks arXiv:1502.00873Google Scholar
  46. 46.
    Swathi K, Kalyana V, Quek CH (2018) Evolutionary based ICA with reference for EEGμRhythm extraction. IEEE Access: 19702 – 19713Google Scholar
  47. 47.
    Sylvia W, Kamalaharidharini T (2017) Robust face recognition and classification system based on SIFT and DCP techniques in image processing. International Conference on Intelligent Computing and Control (I2C2): 1–8Google Scholar
  48. 48.
    Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: closing the gap to human-level performance in face verification. CVPR: 1701–1708Google Scholar
  49. 49.
    Teoh A, Goh A, Ngo D (2006) Random multispace quantization as an analytic mechanism for biohashing of biometric and random identity inputs. IEEE Trans Pattern Anal Mach Intell 28:1892–1901CrossRefGoogle Scholar
  50. 50.
    Theodoridis S, Koutroumbas K (2008) Pattern recognition. Fourth edition. Academic Press, 4th editionGoogle Scholar
  51. 51.
    Vinay A, Kumar C, Shenoy R (2015) ORB-PCA based feature extraction technique for face recognition. Second International Symposium on Computer Vision and the Internet: 614–621CrossRefGoogle Scholar
  52. 52.
    Wright J, Yang A, Ganesh A, Sastry S, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Machine Intell 2:210–227CrossRefGoogle Scholar
  53. 53.
    Xiangyuan L, Ma A, Pong Y, Rama C (2015) Joint sparse representation and robust feature-level fusion for multi-Cue visual tracking. IEEE Trans Image Process 24(12):5826–5841MathSciNetCrossRefGoogle Scholar
  54. 54.
    Xiangyuan L, Shengping Z, Pong Y (2016) Robust joint discriminative feature learning for visual tracking. Twenty-fifth international joint conference on artificial Intelligence: 3403–3410Google Scholar
  55. 55.
    Xiangyuan L, Mang Y, Shengping Z, Pong Y (2018) Robust collaborative discriminative learning for RGB-infrared tracking. The thirty-second AAAI conference on artificial Intelligence: 7008–7015Google Scholar
  56. 56.
    Xiangyuan L, Shengping Z, Pong Y, Rama C (2018) Learning common and feature-specific patterns: a novel multiple-sparse-representation-based tracker. IEEE Trans Image Process 27(4):2022–2037MathSciNetCrossRefGoogle Scholar
  57. 57.
    Xiangyuan L, Mang Y, Rui Sh, Bineng Zh, Pong Y, Huiyu Z (2019) Learning modality-consistency feature templates: a robust RGB-infrared tracking system. IEEE Trans Industrial ElectronicsGoogle Scholar
  58. 58.
    Xiaolin X, Yicong Z (2018) Two-dimensional quaternion PCA and sparse PCA. IEEE Transactions on Neural Networks and Learning Systems: 1–15Google Scholar
  59. 59.
    Xin Q, Gang Z, Yantao L, Ge P (2012) RadioSense: Exploiting Wireless Communication Patterns for Body Sensor Network Activity Recognition. IEEE 33rd Real-Time Systems SymposiumGoogle Scholar
  60. 60.
    Xin Q, Matthew K, Gang Z, Yantao L, Zhen R (2013) AdaSense: adapting sampling rates for activity recognition in body sensor networks. IEEE RTASGoogle Scholar
  61. 61.
    Yakopcic C, Alom M, Taha T (2017) Extremely parallel memristor crossbar architecture for convolutionalneural network implementation. International Joint Conference on Neural Networks (IJCNN): 1696–1703Google Scholar
  62. 62.
    Yuan Z, Liu Y, Yue J, Li J, Yang H (2017) CORAL: Coarse-grained reconfigurable architecture for Convolutional Neural Networks. IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED): 1–6Google Scholar
  63. 63.
    Zhang X, Ren Sh, Sun J (2016) Deep residual learning for image recognition. IEEE conference on computer vision and pattern recognition: 770–778Google Scholar
  64. 64.
    Zheng Y, Pal D, Savvides M (2018) Ring loss: convex feature normalization for face recognition. CVPRGoogle Scholar
  65. 65.
    Zhou E, Cao Z, Yin Q (2015) Naive-deep face recognition: touching the limit of lfw benchmark or not? arXiv preprint arXiv:1501.04690Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Essam Abdellatef
    • 1
    Email author
  • Nabil A. Ismail
    • 2
  • Salah Eldin S. E. Abd Elrahman
    • 2
  • Khalid N. Ismail
    • 3
    • 4
  • Mohamed Rihan
    • 5
  • Fathi E. Abd El-Samie
    • 5
  1. 1.Electronics and Communication DepartmentDelta Academy for EngineeringMansouraEgypt
  2. 2.Department of Computer Science and Engineering, Faculty of Electronic EngineeringMenoufia UniversityMenoufiaEgypt
  3. 3.Department of Computer ScienceDurham UniversityDurhamUK
  4. 4.Information Technology Department, Faculty of Computers and InformationMenoufia UniversityMenoufiaEgypt
  5. 5.Department of Electronics and Electrical Communications Engineering, Faculty of Electronic EngineeringMenoufia UniversityMenoufiaEgypt

Personalised recommendations