Abstract
We present a novel non-invasive neural network based three layered system for detecting fatigue by analyzing facial expressions evaluated using the Facial Action Coding System. We analyze 16 Action Units pertaining to eye and mouth regions of the face. We define an Action Units map containing Action Unit intensity levels for each frame in the video sequence and we analyze this map in a pattern recognition task via a feed-forward neural network. We show that emotion-induced frontal face recordings offer more information in the training stage, while for testing stage the random dataset can be used with no major impact on accuracy, specificity and sensitivity. We obtain over 88% accuracy in intra-subject tests and over 83% for inter-subject tests and we show that our system surpasses the state-of-the-art in terms of accuracy, specificity, sensitivity and response time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Koon, L.Y., Suandi, S.A.: AU measurements from cascaded adaboost for driver drowsiness detection. In: 2013 8th IEEE Conference on Industrial Electronics and Applications (ICIEA), June 2013
Chen, J., et al.: Fatigue detection based on facial images processed difference algorithm. In: 2017 13th IASTED International Conference on Biomedical Engineering (BioMed), February 2017
Kawamura, R., Takemura, N., Sato, K.: Mental fatigue estimation based on luminance changes in facial images. In: IEEE International Symposium on Systems Integration (SI), February 2017
Haque, M.A., Irani, R., Nasrollahi, K., Moeslund, T.B.: Facial video-based detection of physical fatigue for maximal muscle activity. IET Comput. Vis. 10(4), 323–329 (2016)
Giao, Y., Zeng, K., Xu, L., Yin, X.: A smartphone-based driver fatigue detection using fusion of multiple real-time facial features. In: 2016 13th IEEE Annual Consumer Communications and Networking Conference (CCNC), March 2016
Tayibnapis, I.R., Koo, D.Y., Choi, M.K., Kwon, S.: A novel driver fatigue monitoring using optical imaging of face on safe driving system. In: 2016 International Conference on Control, Electronics, Renewable Energy and Communications (ICCEREC), January 2017
Zhang, Y., Hua, C.: Driver fatigue recognition based on facial expression analysis using local binary patterns. Optik – Int. J. Light Electron Opt. 126(23), 4501–4505 (2015)
Ekman, P., Friesen, W.V.: Facial Action Coding System: Investigator’s Guide. Consulting Psychologists Press, Palo Alto (1978)
Mikhail, M., Kaliouby, R.E.: Detection of asymmetric eye action units in spontaneous videos. In: 2009 16th IEEE International Conference on Image Processing (ICIP), Cairo, pp. 3557–3560 (2009)
Liu, X., Cheung, Y.M., Li, M., Liu, H.: A lip contour extraction method using localized active contour model with automatic parameter selection. In: 20th International Conference on Pattern Recognition (ICPR), pp. 4332–4335, August 2010
Tian, Y., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 97–115 (2001)
Pantic, M., Valstar, M.F., Rademaker, R.: Web-based database for facial expression analysis. In: Maat, L. (ed.) Proceedings of IEEE International Conference on Multimedia and Expo (ICME 2005), pp. 317–321, July 2005
Xiaoyuang, L., Bin, Q., Lu, W.: A new improved BP neural network algorithm. In: 2nd International Conference on Intelligent Computation Technology and Automation, pp. 19–22, October 2009
Baveye, Y., Dellandrea, E., Chamaret, C., Chen, L.: LIRIS-ACCEDE: a video database for affective content analysis. IEEE Trans. Affect. Comput. 6(1), 43–55 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Gavrilescu, M., Vizireanu, N. (2018). Neural Network Based Architecture for Fatigue Detection Based on the Facial Action Coding System. In: Fratu, O., Militaru, N., Halunga, S. (eds) Future Access Enablers for Ubiquitous and Intelligent Infrastructures. FABULOUS 2017. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 241. Springer, Cham. https://doi.org/10.1007/978-3-319-92213-3_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-92213-3_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-92212-6
Online ISBN: 978-3-319-92213-3
eBook Packages: Computer ScienceComputer Science (R0)