Face Detection in Thermal Images with YOLOv3

  • Gustavo SilvaEmail author
  • Rui Monteiro
  • André Ferreira
  • Pedro Carvalho
  • Luís Corte-Real
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11845)


The automotive industry is currently focusing on automation in their vehicles, and perceiving the surroundings of an automobile requires the ability to detect and identify objects, events and persons, not only from the outside of the vehicle but also from the inside of the cabin. This constitutes relevant information for defining intelligent responses to events happening on both environments. This work presents a new method for in-vehicle monitoring of passengers, specifically the task of real-time face detection in thermal images, by applying transfer learning with YOLOv3. Using this kind of imagery for this purpose brings some advantages, such as the possibility of detecting faces during the day and in the dark without being affected by illumination conditions, and also because it’s a completely passive sensing solution. Due to the lack of suitable datasets for this type of application, a database of in-vehicle images was created, containing images from 38 subjects performing different head poses and at varying ambient temperatures. The tests in our database show an AP50 of 99.7% and an AP of 78.5%.


Thermal imaging Face detection Computer vision Deep learning YOLOv3 Transfer learning 


  1. 1.
    Basbrain, A.M., Gan, J.Q., Clark, A.: Accuracy enhancement of the viola-jones algorithm for thermal face detection. In: Huang, D.-S., Hussain, A., Han, K., Gromiha, M.M. (eds.) ICIC 2017. LNCS (LNAI), vol. 10363, pp. 71–82. Springer, Cham (2017). Scholar
  2. 2.
    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition (2009).
  3. 3.
    Korukçu, M.Z., Kilic, M.: The usage of IR thermography for the temperature measurements inside an automobile cabin. Int. Commun. Heat Mass Transf. 36(8), 872–877 (2009). Scholar
  4. 4.
    Kwásniewska, A., Rumiński, J., Rad, P.: Deep features class activation map for thermal face detection and tracking. In: Proceedings - 2017 10th International Conference on Human System Interactions, HSI 2017, pp. 41–47 (2017).
  5. 5.
    Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). Scholar
  6. 6.
    Nonaka, Y., Yoshida, D., Kitamura, S., Yokota, T., Hasegawa, M., Ootsu, K.: Monocular color-IR imaging system applicable for various light environments. In: 2018 IEEE International Conference on Consumer Electronics (ICCE), pp. 1–5. IEEE, Las Vegas (2018).
  7. 7.
    Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on feature distributions. Pattern Recogn. 29(1), 51–59 (1996). Scholar
  8. 8.
    Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement (2018).
  9. 9.
    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2015).
  10. 10.
    Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004). Scholar
  11. 11.
    Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 23(10), 1499–1503 (2016). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Gustavo Silva
    • 1
    Email author
  • Rui Monteiro
    • 1
  • André Ferreira
    • 2
  • Pedro Carvalho
    • 3
  • Luís Corte-Real
    • 1
    • 3
  1. 1.Faculty of EngineeringUniversity of PortoPortoPortugal
  2. 2.Bosch Car Multimedia Portugal, S.A.BragaPortugal
  3. 3.INESC TECPortoPortugal

Personalised recommendations