The hyperspectral imaging, capturing discriminative information across a series of spectrum bands, leads to building a robust face recognition system. Motivated by the success of deep convolutional network and transferring learning, this paper proposed an end-to-end hyperspectral face recognition model based on a light Convolutional Neural Network (CNN) and transfer learning. To boost the performance of hyperspectral face recognition, the Max-Feature-Map (MFM) activation function and fine-tuning (structure regularization(L2SP) or attention feature distillation (AFD)) are introduced to optimize the deep network, which will learn the fine feature representation across different bands. Especially, this method incorporates regularization and AFD cooperation into the transfer learning strategy on the visible face data. By feeding back hyperspectral images to the pretrained light CNN network, we can design an end-to end model that can leverage the generalization ability for hyperspectral face images. Finally, the entire model is trained and verified on the PolyU-HSFD, CMU, and UWA hyperspectral face datasets using the associated standard evaluation protocols. Experimental results demonstrate that the improved Light CNN network can get good representations of hyperspectral face features and the joint training with a combination of L2SP and AFD exhibits better recognition performance than the state-of-the-art methods based on other deep networks.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Allen D (2016) An overview of spectral imaging of human skin toward face recognition. In: Face recognition across the imaging Spectrum. Springer, pp 1–19
Aman G, Hasan D (2018) 3D discrete wavelet transform-based feature extraction for hyperspectral face recognition. IET Biometrics 7(1):49–55
Azizpour H, Razavian A, Sullivan J (2015) From generic to specific deep representations for visual recognition. In: IEEE Confer Comput Vision Pattern Recognition. IEEE, pp 36–45
Bo C, Lu H, Wang D (2018) Spectral-spatial K-nearest neighbor approach for hyperspectral image classification. Multimed Tools Appl 77:10419–10436
Cai Z, Vasconcelos N (2018) Cascade R-CNN: delving into high quality object detection. In: IEEE Conference Comput Vis Pattern Recognition. IEEE, pp 6154–6162
Denes L, Metes P, Liu Y (2002) Hyperspectral face database, Robotics Inst., Pittsburgh, PA, Tech. Rep. CMU-RI-TR-02-25
Di W, Zhang L, Zhang D, Pan Q (2010) Studies on hyperspectral face recognition in visible spectrum with feature band selection. IEEE Trans Syst Man Cybern Syst Hum- Part A: Systems and Humans 40(6):1354–1361
Guo Y, Zhang L, Hu Y, He X, Gao J (2016) Ms-celeb-1m: a dataset and benchmark for large-scale face recognition. In: Eur Confer Comput Vis. Springer, pp 87–102
He K, Zhang X, Ren S (2016) Deep residual learning for image recognition. In: IEEE Confer Comput Vis Pattern Recognition. IEEE, pp 770–778
Hu J, Shen L, Albanie S (2017) Squeeze-and-excitation networks. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 7132–7141
Jason Y, Jeff C, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks. In : Proceedings of Twenty-Eighth Confer Neural Inform Process Syst. ACM, 3320–3328
Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. In: Conference and Workshop on Neural Information Processing Systems. ACM, pp 1097–1105
Kruse F (2002) Comparison of AVIRIS and Hyperion for hyperspectral mineral mapping. In : 11th JPL Airborne Geoscience Workshop
Li Z, Hoiem DD (2017) Learning without forgetting. IEEE Trans Pattern Anal Mach Intell 40(12):2935–2947
Li X, Grandvalet Y, Davoine F (2018) Explicit inductive bias for transfer learning with convolutional networks. In: Thirty-fifth international conference on machine learning. ACM, pp 2825–2834
Li, X, Xiong H, Wang H, Rao Y, Liu L, Huan J (2019) DELTA: deep learning transfer using feature map with attention for convolutional networks. In: Proceedings of Int Confer Learn Represent. ICLR, pp 1–13
Lin M, Chen Q, Yan S (2014) Network in network. In: Proceedings of International Conference on Learning Representation. ICLR, pp 1–12
Mollahosseini A, Chan D, Mahoor M (2016) Going deeper in facial expression recognition using deep neural networks. In: IEEE Winter Conference on Applications of Computer Vision. IEEE, pp1–10
Osia N, Bourlai T (2014) A spectral independent approach for physiological and geometric based face recognition in the visible, middle-wave and long-wave infrared bands. Image Vision Comput 32(11):847–859
Pan SQ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359
Pan Z, Healey G, Prasad M, Tromberg B (2003) Face recognition in hyperspectral images. IEEE Trans Pattern Anal Mach Intell 25(12):1552–1560
Patel V, Gopalan R, Li R, Chellappa R (2015) Visual domain adaptation: a survey of recent advances. IEEE Signal Process Mag 32(3):53–69
PolyU-HSFD (1991), www4.comp.polyu.edu.hk/∼biometrics/.
Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Twenty-Ninth Confer Neural Inform Process Syst. ACM, pp 91–99
Sharma S, Diba A (2016) Hyperspectral CNN for image classification &band selection, with application to face recognition. Technical report KUL/ESAT/PSI/1604, KU Leuven, Belgium
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: Proceedings of International Conference on Learning Representation. ICLR2015, pp 1–14
Tan M, Quoc V (2019) Efficient net: rethinking model scaling for convolutional neural networks. In: Int Confer Machine Learn, ACM, pp 6105–6114
Uzair M, Mahmood A, Mian A (2013) Hyperspectral face recognition using 3D-DCT and partial least squares. In: British Machine Vision Conference. British Machine Vision Association, pp 1–7
Uzair M, Mahmood A, Mian A (2015) Hyperspectral face recognition with spatiospectral information fusion and PLS regression. IEEE Trans Image Process 24(3):1127–1137
Veera G, Vasuki S (2017) Clustering based band selection for endmember extraction using simplex growing algorithm in hyperspectral images. Multimedia Tools and Applications 76:8355–8371
Wang K, Gao X, Zhao Y, Li X, Dou D, Xu C (2020) Pay attention to features, transfer learn faster CNNs. In: Proceedings of International Conference on Learning Representations. ICLR2020, pp 1–14
Wu X, He R, Sun Z (2018) A light CNN for deep face representation with Noisy labels. IEEE Trans Information Forensics Secur 13(11):2884–2896
Xie Z, Li Y (2019) Hyperspectral face recognition using block based convolution neural network and AdaBoost band selection. In: Proceedings of International conference on systems and informatics. IEEE, pp 1270–1274
Yi D, Lei Z, Liao S, Li Z (2014) Learning face representation from scratch. arXiv preprint:1411.7923
Yim J, Joo D, Bae J, Kim J (2017) A gift from knowledge distillation: fast optimization, network minimization and transfer learning. In: IEEE Confer Comput Vis Pattern Recognit. IEEE, pp 7130–7138
Zagoruyko S, Komodakis N (2017) Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer. In: Proceedings of International conference on learning representations. ICLR2017, pp 1–13
Zhang X, Pan Z, Lu X (2018) Hyperspectral image classification based on joint spectrum of spatial space and spectral space. Multimed Tools Appl 77:29759–29777
Zhao H, Qi X, Shen X, Shi J, Jia J (2018) ICNet for real-time semantic segmentation on high-resolution images. In: Eur Conference Comput Vis. Springer, pp 405–420
This paper is supported by the National Nature Science Foundation of China (No.61861020), Science &Technology Project of Education Bureau of Jiangxi Province (No. GJJ190578), Jiangxi Province Graduate Innovation Special Fund Project (No. YC2020-S571).
Conflict of interests
The authors declare that they have no conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Xie, Z., Niu, J., Yi, L. et al. Regularization and attention feature distillation base on light CNN for Hyperspectral face recognition. Multimed Tools Appl (2021). https://doi.org/10.1007/s11042-021-10537-4
- Hyperspectral imaging
- Face recognition
- Transfer learning
- Channel attention
- Feature distillation