Skip to main content
SpringerLink
Log in

Evaluation and analysis of ear recognition models: performance, complexity and resource requirements

  • S.I.: Advances in Bio-Inspired Intelligent Systems
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Ear recognition technology has long been dominated by (local) descriptor-based techniques due to their formidable recognition performance and robustness to various sources of image variability. While deep-learning-based techniques have started to appear in this field only recently, they have already shown potential for further boosting the performance of ear recognition technology and dethroning descriptor-based methods as the current state of the art. However, while recognition performance is often the key factor when selecting recognition models for biometric technology, it is equally important that the behavior of the models is understood and their sensitivity to different covariates is known and well explored. Other factors, such as the train- and test-time complexity or resource requirements, are also paramount and need to be consider when designing recognition systems. To explore these issues, we present in this paper a comprehensive analysis of several descriptor- and deep-learning-based techniques for ear recognition. Our goal is to discover weak points of contemporary techniques, study the characteristics of the existing technology and identify open problems worth exploring in the future. We conduct our analysis through identification experiments on the challenging Annotated Web Ears (AWE) dataset and report our findings. The results of our analysis show that the presence of accessories and high degrees of head movement significantly impacts the identification performance of all types of recognition models, whereas mild degrees of the listed factors and other covariates such as gender and ethnicity impact the identification performance only to a limited extent. From a test-time-complexity point of view, the results suggest that lightweight deep models can be equally fast as descriptor-based methods given appropriate computing hardware, but require significantly more resources during training, where descriptor-based methods have a clear advantage. As an additional contribution, we also introduce a novel dataset of ear images, called AWE Extended (AWEx), which we collected from the web for the training of the deep models used in our experiments. AWEx contains 4104 images of 346 subjects and represents one of the largest and most challenging (publicly available) datasets of unconstrained ear images at the disposal of the research community.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Abaza A, Ross A, Hebert C, Harrison MAF, Nixon M (2013) A survey on ear biometrics. ACM Comput Surv 45(2):1–22

    Article  Google Scholar 

  2. Alaraj M, Hou J, Fukami T (2010) A neural network based human identification framework using ear images. In: Proceedings of the international technical conference of IEEE region 10, pp 1595–1600

  3. Arbab-Zavar B, Nixon MS (2008) Robust log-Gabor filter for ear biometrics. In: Proceedings of the international conference on pattern recognition, pp 1–4

  4. Baoqing Z, Zhichun M, Chen J, Jiyuan D (2013) A robust algorithm for ear recognition under partial occlusion. In: Proceedings of the Chinese control conference, pp 3800–3804

  5. Basit A, Shoaib M (2014) A human ear recognition method using nonlinear curvelet feature subspace. Int J Comput Math 91(3):616–624

    Article  MATH  Google Scholar 

  6. Benzaoui A, Hezil N, Boukrouche A (2015) Identity recognition based on the external shape of the human ear. In: Proceedings of the international conference on applied research in computer science and engineering, pp 1–5

  7. Benzaoui A, Kheider A, Boukrouche A (2015) Ear description and recognition using ELBP and wavelets. In: Proceedings of the international conference on applied research in computer science and engineering, pp 1–6

  8. Bourouba H, Doghmane H, Benzaoui A, Boukrouche AH (2015) Ear recognition based on Multi-bags-of-features histogram. In: Proceedings of the international conference on control, engineering information technology, pp 1–6

  9. Bustard JD, Nixon MS (2010) Toward unconstrained ear recognition from two-dimensional images. Trans Syst Man Cybern Part A Syst Hum 40(3):486–494

    Article  Google Scholar 

  10. Chan T-S, Kumar A (2012) Reliable ear identification using 2-D quadrature filters. Pattern Recogn Lett 33(14):1870–1881

    Article  Google Scholar 

  11. Choraś M (2008) Perspective methods of human identification: ear biometrics. Opto-Electron Rev 16(1):85–96

    Article  Google Scholar 

  12. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of the international conference on computer vision and pattern recognition, pp 886–893

  13. Damar N, Fuhrer B (2012) Ear recognition using multi-scale histogram of oriented gradients. In: Proceedings of the conference on intelligent information hiding and multimedia signal processing, pp 21–24

  14. Daugman J (1985) Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. J Opt Soc Am A 2(7):1160–1169

    Article  Google Scholar 

  15. Dewi K, Yahagi T (2006) Ear photo recognition using scale invariant keypoints. In: Proceedings of the computational intelligence, pp 253–258

  16. Dodge S, Mounsef J, Karam L (2018) Unconstrained ear recognition using deep neural networks. IET Biom 7:207–214

    Article  Google Scholar 

  17. Eyiokur FI, Yaman D, Ekenel HK (2018) Domain adaptation for ear recognition using deep convolutional neural networks. IET Biom 7:199–206

    Article  Google Scholar 

  18. Earnest H, Segundo P, Sarkar S (2018) Employing fusion of learned and handcrafted features for unconstrained ear recognition. IET Biom 7:215–223

    Article  Google Scholar 

  19. Emeršič Ž, Meden B, Peer P, Štruc V (2017) Covariate analysis of descriptor-based ear recognition techniques. In: 2017 international conference and workshop on bioinspired intelligence (IWOBI), pp 1–9

  20. Emeršič Ž, Peer P (2015) Ear biometric database in the wild. In: 2015 4th international work conference on bioinspired intelligence (IWOBI), pp 27–32

  21. Emeršič Ž, Peer P (2015) Toolbox for ear biometric recognition evaluation. In: EUROCON 2015—international conference on computer as a tool (EUROCON), IEEE, pp 1–6

  22. Emeršič Ž, Štepec D, Štruc V, Peer P (2017) Training convolutional neural networks with limited training data for ear recognition in the wild. In: Proceedings of the 12th IEEE international conference on automatic face and gesture (FG 2017)

  23. Emeršič Ž, Štepec D, Štruc V, Peer P, George A, Ahmad A, Omar E, Boult TE, Safdari R, Zhou Y, Zafeiriou S, Yaman D, Eyiokur FI, Ekenel HK (2017) The unconstrained ear recognition challenge. In: International joint conference on biometrics (IJCB)

  24. Emeršič Ž, Štruc V, Peer P (2017) Ear recognition: more than a survey. Neurocomputing 255:26–39

    Article  Google Scholar 

  25. Grm K, Štruc V, Artiges A, Caron M, Ekenel HK (2017) Strengths and weaknesses of deep learning models for face recognition against image degradations. IET Biom 7:81–89

    Article  Google Scholar 

  26. Guo Y, Xu Z (2008) Ear recognition using a new local matching approach. In: Proceedings of the international conference on image processing, pp 289–292

  27. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  28. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision. Springer, Berlin, pp 630–645

  29. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) Squeezenet: Alexnet-level accuracy with 50x fewer parameters and 0.5 mb model size. arXiv preprint arXiv:1602.07360,

  30. Kannala J, Rahtu E (2012) BSIF: Binarized statistical image features. In: Proceedings of the international conference on pattern recognition, pp 1363–1366

  31. Križaj J, Štruc V, Pavešic N (2010) Adaptation of SIFT features for robust face recognition. In: Proceedings of the image analysis and recognition. Springer, New York, pp 394–404

  32. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  33. Kumar A, Wu C (2012) Automated human identification using ear imaging. Pattern Recogn 45(3):956–968

    Article  Google Scholar 

  34. Kumar A, Zhang D (2007) Ear authentication using log-gabor wavelets. In: Proceedings of the symposium on defense and security. International society for optics and photonics, p 65390A

  35. Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  36. Meraoumia A, Chitroub S, Bouridane A (2015) An automated ear identification system using Gabor filter responses. In: Proceedings of the international conference on new circuits and systems, pp 1–4

  37. Morales A, Ferrer M, Diaz-Cabrera M, Gonzalez E (2013) Analysis of local descriptors features and its robustness applied to ear recognition. In: Proceedings of the international carnahan conference on security technology, pp 1–5

  38. Nanni L, Lumini A (2009) Fusion of color spaces for ear authentication. Pattern Recogn 42(9):1906–1913

    Article  MATH  Google Scholar 

  39. Ojansivu V, Heikkilä J (2008) Blur insensitive texture classification using local phase quantization. In: Image and signal processing, Springer, New York, pp 236–243

  40. Ojansivu V, Rahtu E, Heikkilä J (2008) Rotation invariant local phase quantization for blur insensitive texture analysis. In: Proceedings of the international conference on pattern recognition, pp 1–4

  41. Pflug A, Busch C (2012) Ear biometrics: a survey of detection, feature extraction and recognition methods. Biometrics 1(2):114–129

    Google Scholar 

  42. Pflug A, Busch C, Ross A (2014) 2D ear classification based on unsupervised clustering. In: Proceedings of the international joint conference on biometrics, pp 1–8

  43. Pflug A, Paul PN, Busch C (2014) A comparative study on texture and surface descriptors for ear biometrics. In: Proceedings of the international Carnahan conference on security technology, pp 1–6

  44. Pflug A, Wagner J, Rathgeb C, Busch C (2014) Impact of severe signal degradation on ear recognition performance. In: 2014 37th international convention on information and communication technology, electronics and microelectronics (MIPRO), pp 1342–1347

  45. Pietikäinen M, Hadid A, Zhao G, Ahonen T (2011) Computer vision using local binary patterns. Computational imaging and vision. Springer, New York

    Book  Google Scholar 

  46. Prakash S, Gupta P (2013) An efficient ear recognition technique invariant to illumination and pose. Telecommun Syst 52(3):1435–1448

    Article  Google Scholar 

  47. Purkait R (2015) Role of external ear in establishing personal identity—a short review. Austin J Forensic Sci Criminol 2(2):1–5

    MathSciNet  Google Scholar 

  48. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  49. Štruc V, Gajšek R, Pavesic N (2009) Principal Gabor filters for face recognition. In: Proceedings of the conference on biometrics: theory, applications and systems, pp 1–6

  50. Štruc V, Pavesic N (2009) Gabor-based kernel partial-least-squares discrimination features for face recognition. EURASIP J Adv Signal Process 20(1):115–138

    MATH  Google Scholar 

  51. Štruc V, Pavešic N (2010) The complete gabor-fisher classifier for robust face recognition. EURASIP J Adv Signal Process 1–26:2010

    MATH  Google Scholar 

  52. Vu N-S, Caplier A (2010) Face recognition with patterns of oriented edge magnitudes. In: European conference on computer vision, pp 313–326

  53. Xiaoyun W, Weiqi Y (2009) Human ear recognition based on block segmentation. In: Proceedings of the international conference on cyber-enabled distributed computing and knowledge discovery, pp 262–266

  54. Xie Z, Mu Z (2008) Ear recognition using LLE and IDLLE algorithm. In: Proceedings of the international conference on pattern recognition, pp 1–4

  55. Zhang Y, Mu Z, Yuan L, Yu C (2018) Ear verification under uncontrolled conditions with convolutional neural networks. IET Biom 7. https://ieeexplore.ieee.org/abstract/document/8340919/

  56. Zhang Z, Liu H (2008) Multi-view ear recognition based on B-Spline pose manifold construction. In: Proceedings of the world congress on intelligent control and automation

Download references

Acknowledgements

This research was supported in parts by the ARRS (Slovenian Research Agency) Research Program P2-0250 (B) Metrology and Biometric Systems, the ARRS Research Program P2-0214 (A) Computer Vision. The authors thank NVIDIA for donating the Titan Xp GPU that was used in the experiments.

Author information

Authors and Affiliations

  1. Faculty of Computer and Information Science, University of Ljubljana, Večna pot 113, 1000, Ljubljana, Slovenia

    Žiga Emeršič, Blaž Meden & Peter Peer

  2. Faculty of Electrical Engineering, University of Ljubljana, Tržaška 25, 1000, Ljubljana, Slovenia

    Vitomir Štruc

Authors
  1. Žiga Emeršič
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Blaž Meden
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Peer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vitomir Štruc
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Žiga Emeršič.

Ethics declarations

Conflict of interest

We warrant that the article has not received prior publication, is not under consideration for publication elsewhere and is an original work. On behalf of all co-authors, the corresponding author bears full responsibility for the submission. The authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Emeršič, Ž., Meden, B., Peer, P. et al. Evaluation and analysis of ear recognition models: performance, complexity and resource requirements. Neural Comput & Applic 32, 15785–15800 (2020). https://doi.org/10.1007/s00521-018-3530-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-018-3530-1

Keywords

Search

Navigation