Skip to main content
SpringerLink
Log in

Thermal Face Recognition in Unconstrained Environments Using Histograms of LBP Features

  • Chapter
  • First Online:
Local Binary Patterns: New Variants and Applications

Part of the book series: Studies in Computational Intelligence ((SCI,volume 506))

Abstract

Several studies have shown that the use of thermal images can solve limitations of visible spectrum based face recognition methods operating in unconstrained environments. The recognition of faces in the thermal domain can be tackled using the histograms of Local Binary Pattern (LBP) features method. The aim of this work is to analyze the advantages and limitations of this method by means of a comparative study against other methods. The analyzed methods were selected by considering their performance in former comparative studies, in addition to being real-time—10 fps or more—to require just one image per person, and to being fully online (no requirements of offline enrollment). Thus, in the analysis the following local-matching based methods are considered: Gabor Jet Descriptors (GJD), Weber Linear Discriminant (WLD) and Local Binary Pattern (LBP). The methods are compared using the UCHThermalFace database. The use of this database allows evaluating the methods in real-world conditions that include natural variations in illumination, indoor/outdoor setup, facial expression, pose, accessories, occlusions, and background. In addition, the fusion of some variants of the methods was evaluated. The main conclusions of the comparative study are: (i) All analyzed methods perform very well under the conditions in which they were evaluated, except for the case of GJD that has low performance in outdoor setups; (ii) the best tradeoff between high recognition rate and fast processing speed is obtained by LBP-based methods; and (iii) fusing some methods or their variants improve the results up to 5 %.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The UCHThermalFace database is available for download at http://vision.die.uchile.cl/dbThermal/.

  2. 2.

    http://www.flir.com/cvs/cores/uncooled/products/tau/

  3. 3.

    In Tables 9, 10 and 11, unlike in previous tables, the average top-1 recognition results do consider using the set R6 as a test set.

References

  1. Abidi, B., Huq, S., Abidi, M.: Fusion of visual, thermal, and range as a solution to illumination and pose restrictions in face recognition. In: Proceedings of IEEE Carnahan Conference on Security Technology, pp. 325–330, Albuquerque, NM (2004)

    Google Scholar 

  2. Ahmad, J., Ali, U., Qureshi, R.J.: Fusion of thermal and visual images for efficient face recognition using gabor filter. In: The 4th ACS/IEEE International Conference on Computer Systems and Applications, pp. 135–139, March 8–11. Dubai/Sharjah, UAE (2006)

    Google Scholar 

  3. Ahonen, T., Hadid, A., Pietikainen, M.: Face Description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)

    Google Scholar 

  4. Akhloufi, M., Bendada, A.: Thermal faceprint: a new thermal face signature extraction for infrared face recognition, In: CRV 2008: Fifth Canadian Conference on Computer and Robot Vision (Windsor, Ontario), pp. 269–272, 28–30 May 2008

    Google Scholar 

  5. Bebis, G., Gyaourova, A., Singh, S., Pavlidis, I.: Face recognition by fusing thermal infrared and visible imagery. Image Vis. Comput. 24(7), 727–742 (2006)

    Google Scholar 

  6. Bhowmik, M.K., Bhattacharjee, D., Nasipuri, M., Basu, D.K., Kundu, M.: Optimum fusion of visual and thermal face images for recognition. In: Sixth International Conference on Information Assurance and Security (IAS 2010), pp. 311–316. IEEE Intelligent Transportation Systems Society, Atlanta, USA, 23–25 August 2010

    Google Scholar 

  7. Buddharaju, P., Pavlidis, I.: Multi-spectral face recognition—fusion of visual imagery with physiological information. In: Hammoud, R.I., Abidi, B.R., Abidi, M.A. (eds.) Face Biometrics for Personal Identification: Multi-Sensory Multi-Modal Systems, pp. 91–108. Springer, Berlin (2007)

    Google Scholar 

  8. Buddharaju, P., Pavlidis, I., Manohar, C.: Face Recognition Beyond the Visible Spectrum, Advances in Biometrics: Sensors, Algorithms and Systems, pp. 157–180. Springer, Berlin (2007)

    Google Scholar 

  9. Buddharaju, P., Pavlidis, I., Kakadiaris, I.: Physiology-based face recognition. In: Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, pp. 354–359. Lake Como, Italy (2005)

    Google Scholar 

  10. Buddharaju, P., Pavlidis, I., Kakadiaris, I.: Pose-invariant physiological face recognition in the thermal infrared spectrum. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 53–60. New York, USA (2006)

    Google Scholar 

  11. Chen, J., Shan, S., He, Ch., Zhao, G., Pietikäinen, M., Chen, ChX, Gao, W.: WLD: a robust local image descriptor. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1705–1720 (2010)

    Google Scholar 

  12. Chen, X., Flynn, P., Bowyer, K.W.: PCA-based face recognition in infrared imagery: baseline and comparative studies. In: IEEE International Workshop on Analysis and Modeling of Faces and Gestures, pp. 127–134. Nice, France (2003)

    Google Scholar 

  13. Cho, S.Y., Wang, L., Ong, W.L.: Thermal imprint feature analysis for face recognition. In: IEEE International Symposium on Industrial Electronics 2009, ISIE 2009, pp. 1875–1880 (2009)

    Google Scholar 

  14. Desa, S., Hati, S.: IR and visible face recognition using fusion of kernel based features. In: The 19th International Conference on Pattern Recognition (ICPR 2008), pp. 1–4. Tampa, Florida, USA, 8–11 December 2008

    Google Scholar 

  15. Equinox, 2011. Equinox Database. http://www.equinoxsensors.com/products/HID.html. Accessed July 2011

  16. He, M., Horng, S.-J., Fan, P., Run, R.-S., Chen, R.-J., Lai, J.-L., Khan, M., Sentosa, K.: Performance evaluation of score level fusion in multimodal biometric systems. Pattern Recognit. 43(5), 1789–1800 (2010)

    Google Scholar 

  17. Hermosilla, G.,Loncomilla, P., Ruiz-del-Solar, J.: Thermal Face Recognition using Local Interest Points and Descriptors for HRI Applications. Lecture Notes in Computer Science, vol. 6556 (RoboCup, Symposium 2010), pp. 25–35

    Google Scholar 

  18. Hermosilla, G., Ruiz-del-Solar, J., Verschae, R., Correa, M.: Face recognition using thermal infrared images for human-robot interaction applications: a comparative study. In: The 6th IEEE Latin American Robotics Symposium—LARS 2009, Valparaíso, Chile (CD Proceedings), 29–30 October

    Google Scholar 

  19. Hermosilla, G., Ruiz-del-Solar, J., Verschae, R., Correa, M.: A comparative study of thermal face recognition methods in unconstrained environments. Pattern Recognit. 45(7), 2445–2459. ISSN 0031–3203 (July 2012). doi:10.1016/j.patcog.2012.01.001

  20. Kwon, O.K., Kong, S.G.: Multiscale fusion of visual and thermal images for robust face recognition. In: Proceedings of IEEE International Conference on Computer Intelligence for Homeland Security and Personal Safety, vol. IV, pp. 112–116, FL, March 2005

    Google Scholar 

  21. Lan, W.: Face recognition system based on spatial constellation model and support vector machine. Master’s Thesis, Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology (2010)

    Google Scholar 

  22. Lategahn, H., Gross, S., Stehle, T., Aach, T.: Texture classification by modeling joint distributions of local patterns with Gaussian mixtures. IEEE Trans. Image Processing 19(6), 1548–1557 (2010)

    Google Scholar 

  23. Lei, Z., Li, S.Z.: Fast multi-scale local phase quantization histogram for face recognition. Pattern Recognit. Lett. 33(13), 1761–1767. ISSN 0167–8655 (2012). doi:10.1016/j.patrec.2012.06.005

  24. Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Google Scholar 

  25. Mendez, H., San Martín, C., Kittler, J., Plasencia, Y., García, E.: Face recognition with LWIR imagery using local binary patterns. LNCS, vol. 5558, pp. 327–336 (2009)

    Google Scholar 

  26. Nanni, L., Lumini, A., Brahnam, S.: High performance set of features for biometric data. Int. J. Autom. Ident. Technol 2(1), 1–7 (2010)

    Google Scholar 

  27. Narendra, P.: Reference-free nonuniformity compensation for IR imaging arrays. Proc. SPIE 252, 10–17 (1980)

    Google Scholar 

  28. Narendra, P., Foss, N.: Shutterless fixed pattern noise correction for infrared imaging arrays. Proc. SPIE 282, 44–51 (1981)

    Google Scholar 

  29. Pop, F.M., Gordan, M., Florea, C., Vlaicu, A.: Fusion based approach for thermal and visible face recognition under pose and expressivity variation. In: Roedunet Int. Conf. (RoEduNet), pp. 61–66 (2010)

    Google Scholar 

  30. Ruiz-del-Solar, J., Navarrete, P.: Eigenspace-based face recognition: a comparative study of different approaches. IEEE Trans. Syst. Man Cybernet.Part C 35(3), 315–325 (2005)

    Google Scholar 

  31. Ruiz-del-Solar, J., Quinteros, J.: Illumination compensation and normalization in eigenspace-based face recognition: a comparative study of different pre-processing approaches. Pattern Recognit. Lett. 29(14), 1966–1979 (2008)

    Google Scholar 

  32. Ruiz-del-Solar, J., Verschae, R., Correa, M.: Recognition of faces in unconstrained environments: a comparative study. EURASIP Journal on Advances in Signal Processing, special issue, Recent Advances in Biometric Systems: A Signal Processing Perspective, vol. 2009, Article ID 184617, 19 pages (2009). doi:10.1155/2009/184617

  33. Selinger, A., Socolinsky, D.: Appearance-based facial recognition using visible and thermal imagery: a comparative study. Tech. Rep., Equinox Corporation (2001)

    Google Scholar 

  34. Socolinsky, D., Selinger, A.: A comparative analysis of face recognition performance with visible and thermal infrared imagery. In: Proceedings of the International Conference on Pattern Recognition (ICPR), Quebec, Canada (2002)

    Google Scholar 

  35. Socolinsky, D., Selinger, A.: Thermal face recognition over time. In: Proceedings of the 17th International Conference on Pattern Recognition (ICPR’04), vol. 4, pp. 187–190 (2004)

    Google Scholar 

  36. Socolinsky, D., Wolff, L., Neuheisel, J., Eveland, C.: Illumination invariant face recognition using thermal infrared imagery. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2001)

    Google Scholar 

  37. Tan, X., Chen, S., Zhou, Z.H., Zhang, F.: Face recognition from a single image per person: a survey. Pattern Recognit. 39, 1725–1745 (2006)

    Google Scholar 

  38. Yoshitomi, Y., Sung-Ill K., Kawano, T., Kilazoe, T.: Effect of sensor fusion for recognition of emotional states using voice, face image and thermal image of face. In: Proceedings of 9th IEEE International Workshop on Robot and Human Interactive Communication, pp. 178–183 (2000)

    Google Scholar 

  39. Zou, J., Ji, Q., Nagy, G.: A comparative study of local matching approach for face recognition. IEEE Trans. Image Process. 16(10), 2617–2628 (2007)

    Google Scholar 

Download references

Acknowledgments

This research was partially funded by the FONDECYT-Chile grant 1090250, by the FONDECYT-Chile grant 3120218, and by the Advanced Mining Technology Center.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Universidad de Chile, Av. Tupper 2007, 837-0451 , Santiago, Chile

    Javier Ruiz-del-Solar

  2. Advanced Mining Technology Center, Universidad de Chile, Av. Tupper 2007, 837-0451 , Santiago, Chile

    Javier Ruiz-del-Solar, Rodrigo Verschae & Mauricio Correa

  3. Escuela de Ingeniería Eléctrica, Pontificia Universidad Católica de Valparaíso, Valparaiso, Chile

    Gabriel Hermosilla

Authors
  1. Javier Ruiz-del-Solar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rodrigo Verschae
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gabriel Hermosilla
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mauricio Correa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Javier Ruiz-del-Solar .

Editor information

Editors and Affiliations

  1. Computer Information Systems, Missouri State University, Springfield, USA

    Sheryl Brahnam

  2. Faculty of Education, Science, Technology & Mathematics, University of Canberra, Canberra, Australia

    Lakhmi C. Jain

  3. Departimento di Elettronica e Informatica (DEI), Università di Padova, Padua, Italy

    Loris Nanni

  4. DEIS University of Bologna, Cesena, Italy

    Alessandra Lumini

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Ruiz-del-Solar, J., Verschae, R., Hermosilla, G., Correa, M. (2014). Thermal Face Recognition in Unconstrained Environments Using Histograms of LBP Features. In: Brahnam, S., Jain, L., Nanni, L., Lumini, A. (eds) Local Binary Patterns: New Variants and Applications. Studies in Computational Intelligence, vol 506. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39289-4_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-39289-4_10

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-39288-7

  • Online ISBN: 978-3-642-39289-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation