Advertisement

Evaluation and analysis of ear recognition models: performance, complexity and resource requirements

  • Žiga Emeršič
  • Blaž Meden
  • Peter Peer
  • Vitomir Štruc
S.I.: Advances in Bio-Inspired Intelligent Systems

Abstract

Ear recognition technology has long been dominated by (local) descriptor-based techniques due to their formidable recognition performance and robustness to various sources of image variability. While deep-learning-based techniques have started to appear in this field only recently, they have already shown potential for further boosting the performance of ear recognition technology and dethroning descriptor-based methods as the current state of the art. However, while recognition performance is often the key factor when selecting recognition models for biometric technology, it is equally important that the behavior of the models is understood and their sensitivity to different covariates is known and well explored. Other factors, such as the train- and test-time complexity or resource requirements, are also paramount and need to be consider when designing recognition systems. To explore these issues, we present in this paper a comprehensive analysis of several descriptor- and deep-learning-based techniques for ear recognition. Our goal is to discover weak points of contemporary techniques, study the characteristics of the existing technology and identify open problems worth exploring in the future. We conduct our analysis through identification experiments on the challenging Annotated Web Ears (AWE) dataset and report our findings. The results of our analysis show that the presence of accessories and high degrees of head movement significantly impacts the identification performance of all types of recognition models, whereas mild degrees of the listed factors and other covariates such as gender and ethnicity impact the identification performance only to a limited extent. From a test-time-complexity point of view, the results suggest that lightweight deep models can be equally fast as descriptor-based methods given appropriate computing hardware, but require significantly more resources during training, where descriptor-based methods have a clear advantage. As an additional contribution, we also introduce a novel dataset of ear images, called AWE Extended (AWEx), which we collected from the web for the training of the deep models used in our experiments. AWEx contains 4104 images of 346 subjects and represents one of the largest and most challenging (publicly available) datasets of unconstrained ear images at the disposal of the research community.

Keywords

Ear recognition Covariate analysis Convolutional neural networks Feature extraction 

Notes

Acknowledgements

This research was supported in parts by the ARRS (Slovenian Research Agency) Research Program P2-0250 (B) Metrology and Biometric Systems, the ARRS Research Program P2-0214 (A) Computer Vision. The authors thank NVIDIA for donating the Titan Xp GPU that was used in the experiments.

Compliance with ethical standards

Conflict of interest

We warrant that the article has not received prior publication, is not under consideration for publication elsewhere and is an original work. On behalf of all co-authors, the corresponding author bears full responsibility for the submission. The authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest.

References

  1. 1.
    Abaza A, Ross A, Hebert C, Harrison MAF, Nixon M (2013) A survey on ear biometrics. ACM Comput Surv 45(2):1–22CrossRefGoogle Scholar
  2. 2.
    Alaraj M, Hou J, Fukami T (2010) A neural network based human identification framework using ear images. In: Proceedings of the international technical conference of IEEE region 10, pp 1595–1600Google Scholar
  3. 3.
    Arbab-Zavar B, Nixon MS (2008) Robust log-Gabor filter for ear biometrics. In: Proceedings of the international conference on pattern recognition, pp 1–4Google Scholar
  4. 4.
    Baoqing Z, Zhichun M, Chen J, Jiyuan D (2013) A robust algorithm for ear recognition under partial occlusion. In: Proceedings of the Chinese control conference, pp 3800–3804Google Scholar
  5. 5.
    Basit A, Shoaib M (2014) A human ear recognition method using nonlinear curvelet feature subspace. Int J Comput Math 91(3):616–624CrossRefzbMATHGoogle Scholar
  6. 6.
    Benzaoui A, Hezil N, Boukrouche A (2015) Identity recognition based on the external shape of the human ear. In: Proceedings of the international conference on applied research in computer science and engineering, pp 1–5Google Scholar
  7. 7.
    Benzaoui A, Kheider A, Boukrouche A (2015) Ear description and recognition using ELBP and wavelets. In: Proceedings of the international conference on applied research in computer science and engineering, pp 1–6Google Scholar
  8. 8.
    Bourouba H, Doghmane H, Benzaoui A, Boukrouche AH (2015) Ear recognition based on Multi-bags-of-features histogram. In: Proceedings of the international conference on control, engineering information technology, pp 1–6Google Scholar
  9. 9.
    Bustard JD, Nixon MS (2010) Toward unconstrained ear recognition from two-dimensional images. Trans Syst Man Cybern Part A Syst Hum 40(3):486–494CrossRefGoogle Scholar
  10. 10.
    Chan T-S, Kumar A (2012) Reliable ear identification using 2-D quadrature filters. Pattern Recogn Lett 33(14):1870–1881CrossRefGoogle Scholar
  11. 11.
    Choraś M (2008) Perspective methods of human identification: ear biometrics. Opto-Electron Rev 16(1):85–96Google Scholar
  12. 12.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of the international conference on computer vision and pattern recognition, pp 886–893Google Scholar
  13. 13.
    Damar N, Fuhrer B (2012) Ear recognition using multi-scale histogram of oriented gradients. In: Proceedings of the conference on intelligent information hiding and multimedia signal processing, pp 21–24Google Scholar
  14. 14.
    Daugman J (1985) Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. J Opt Soc Am A 2(7):1160–1169CrossRefGoogle Scholar
  15. 15.
    Dewi K, Yahagi T (2006) Ear photo recognition using scale invariant keypoints. In: Proceedings of the computational intelligence, pp 253–258Google Scholar
  16. 16.
    Dodge S, Mounsef J, Karam L (2018) Unconstrained ear recognition using deep neural networks. IET Biom 7:207–214CrossRefGoogle Scholar
  17. 17.
    Eyiokur FI, Yaman D, Ekenel HK (2018) Domain adaptation for ear recognition using deep convolutional neural networks. IET Biom 7:199–206CrossRefGoogle Scholar
  18. 18.
    Earnest H, Segundo P, Sarkar S (2018) Employing fusion of learned and handcrafted features for unconstrained ear recognition. IET Biom 7:215–223CrossRefGoogle Scholar
  19. 19.
    Emeršič Ž, Meden B, Peer P, Štruc V (2017) Covariate analysis of descriptor-based ear recognition techniques. In: 2017 international conference and workshop on bioinspired intelligence (IWOBI), pp 1–9Google Scholar
  20. 20.
    Emeršič Ž, Peer P (2015) Ear biometric database in the wild. In: 2015 4th international work conference on bioinspired intelligence (IWOBI), pp 27–32Google Scholar
  21. 21.
    Emeršič Ž, Peer P (2015) Toolbox for ear biometric recognition evaluation. In: EUROCON 2015—international conference on computer as a tool (EUROCON), IEEE, pp 1–6Google Scholar
  22. 22.
    Emeršič Ž, Štepec D, Štruc V, Peer P (2017) Training convolutional neural networks with limited training data for ear recognition in the wild. In: Proceedings of the 12th IEEE international conference on automatic face and gesture (FG 2017)Google Scholar
  23. 23.
    Emeršič Ž, Štepec D, Štruc V, Peer P, George A, Ahmad A, Omar E, Boult TE, Safdari R, Zhou Y, Zafeiriou S, Yaman D, Eyiokur FI, Ekenel HK (2017) The unconstrained ear recognition challenge. In: International joint conference on biometrics (IJCB)Google Scholar
  24. 24.
    Emeršič Ž, Štruc V, Peer P (2017) Ear recognition: more than a survey. Neurocomputing 255:26–39CrossRefGoogle Scholar
  25. 25.
    Grm K, Štruc V, Artiges A, Caron M, Ekenel HK (2017) Strengths and weaknesses of deep learning models for face recognition against image degradations. IET Biom 7:81–89CrossRefGoogle Scholar
  26. 26.
    Guo Y, Xu Z (2008) Ear recognition using a new local matching approach. In: Proceedings of the international conference on image processing, pp 289–292Google Scholar
  27. 27.
    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778Google Scholar
  28. 28.
    He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision. Springer, Berlin, pp 630–645Google Scholar
  29. 29.
    Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) Squeezenet: Alexnet-level accuracy with 50x fewer parameters and 0.5 mb model size. arXiv preprint arXiv:1602.07360,
  30. 30.
    Kannala J, Rahtu E (2012) BSIF: Binarized statistical image features. In: Proceedings of the international conference on pattern recognition, pp 1363–1366Google Scholar
  31. 31.
    Križaj J, Štruc V, Pavešic N (2010) Adaptation of SIFT features for robust face recognition. In: Proceedings of the image analysis and recognition. Springer, New York, pp 394–404Google Scholar
  32. 32.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105Google Scholar
  33. 33.
    Kumar A, Wu C (2012) Automated human identification using ear imaging. Pattern Recogn 45(3):956–968CrossRefGoogle Scholar
  34. 34.
    Kumar A, Zhang D (2007) Ear authentication using log-gabor wavelets. In: Proceedings of the symposium on defense and security. International society for optics and photonics, p 65390AGoogle Scholar
  35. 35.
    Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRefGoogle Scholar
  36. 36.
    Meraoumia A, Chitroub S, Bouridane A (2015) An automated ear identification system using Gabor filter responses. In: Proceedings of the international conference on new circuits and systems, pp 1–4Google Scholar
  37. 37.
    Morales A, Ferrer M, Diaz-Cabrera M, Gonzalez E (2013) Analysis of local descriptors features and its robustness applied to ear recognition. In: Proceedings of the international carnahan conference on security technology, pp 1–5Google Scholar
  38. 38.
    Nanni L, Lumini A (2009) Fusion of color spaces for ear authentication. Pattern Recogn 42(9):1906–1913CrossRefzbMATHGoogle Scholar
  39. 39.
    Ojansivu V, Heikkilä J (2008) Blur insensitive texture classification using local phase quantization. In: Image and signal processing, Springer, New York, pp 236–243Google Scholar
  40. 40.
    Ojansivu V, Rahtu E, Heikkilä J (2008) Rotation invariant local phase quantization for blur insensitive texture analysis. In: Proceedings of the international conference on pattern recognition, pp 1–4Google Scholar
  41. 41.
    Pflug A, Busch C (2012) Ear biometrics: a survey of detection, feature extraction and recognition methods. Biometrics 1(2):114–129Google Scholar
  42. 42.
    Pflug A, Busch C, Ross A (2014) 2D ear classification based on unsupervised clustering. In: Proceedings of the international joint conference on biometrics, pp 1–8Google Scholar
  43. 43.
    Pflug A, Paul PN, Busch C (2014) A comparative study on texture and surface descriptors for ear biometrics. In: Proceedings of the international Carnahan conference on security technology, pp 1–6Google Scholar
  44. 44.
    Pflug A, Wagner J, Rathgeb C, Busch C (2014) Impact of severe signal degradation on ear recognition performance. In: 2014 37th international convention on information and communication technology, electronics and microelectronics (MIPRO), pp 1342–1347Google Scholar
  45. 45.
    Pietikäinen M, Hadid A, Zhao G, Ahonen T (2011) Computer vision using local binary patterns. Computational imaging and vision. Springer, New YorkCrossRefGoogle Scholar
  46. 46.
    Prakash S, Gupta P (2013) An efficient ear recognition technique invariant to illumination and pose. Telecommun Syst 52(3):1435–1448CrossRefGoogle Scholar
  47. 47.
    Purkait R (2015) Role of external ear in establishing personal identity—a short review. Austin J Forensic Sci Criminol 2(2):1–5Google Scholar
  48. 48.
    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
  49. 49.
    Štruc V, Gajšek R, Pavesic N (2009) Principal Gabor filters for face recognition. In: Proceedings of the conference on biometrics: theory, applications and systems, pp 1–6Google Scholar
  50. 50.
    Štruc V, Pavesic N (2009) Gabor-based kernel partial-least-squares discrimination features for face recognition. EURASIP J Adv Signal Process 20(1):115–138zbMATHGoogle Scholar
  51. 51.
    Štruc V, Pavešic N (2010) The complete gabor-fisher classifier for robust face recognition. EURASIP J Adv Signal Process 1–26:2010zbMATHGoogle Scholar
  52. 52.
    Vu N-S, Caplier A (2010) Face recognition with patterns of oriented edge magnitudes. In: European conference on computer vision, pp 313–326Google Scholar
  53. 53.
    Xiaoyun W, Weiqi Y (2009) Human ear recognition based on block segmentation. In: Proceedings of the international conference on cyber-enabled distributed computing and knowledge discovery, pp 262–266Google Scholar
  54. 54.
    Xie Z, Mu Z (2008) Ear recognition using LLE and IDLLE algorithm. In: Proceedings of the international conference on pattern recognition, pp 1–4Google Scholar
  55. 55.
    Zhang Y, Mu Z, Yuan L, Yu C (2018) Ear verification under uncontrolled conditions with convolutional neural networks. IET Biom 7. https://ieeexplore.ieee.org/abstract/document/8340919/
  56. 56.
    Zhang Z, Liu H (2008) Multi-view ear recognition based on B-Spline pose manifold construction. In: Proceedings of the world congress on intelligent control and automationGoogle Scholar

Copyright information

© The Natural Computing Applications Forum 2018

Authors and Affiliations

  1. 1.Faculty of Computer and Information ScienceUniversity of LjubljanaLjubljanaSlovenia
  2. 2.Faculty of Electrical EngineeringUniversity of LjubljanaLjubljanaSlovenia

Personalised recommendations