Comparative Analysis of Machine Learning Techniques in Assessing Cognitive Workload
Cognitive workload refers to the amount of mental capacity that is stored in working memory and remains a vital yet challenging aspect of many human factors applications. This paper presents a comprehensive analysis of alternatives for different machine learning techniques used for assessing mental workload. This methodology consists of five techniques and four datasets. Each dataset consists of physiological factors such as electrodermal activity (EDA) or heart rate data as well as subjective factors. When evaluating our techniques, we compare accuracy, runtime, and F1 score across multiple different method-specific experimental parameters. Ultimately, the results indicate that ANNs perform best on datasets with large numbers of inputs and classification options, while decision trees are most ideal under a lower number of output possibilities. By understanding these individual strengths and weaknesses, we can ultimately improve the balance of ideal human performance through early detection of cognitive overload or underload.
KeywordsCognitive workload Machine learning Artificial neural networks Support vector machines K-nearest neighbors Decision trees Random forests
This research is supported by the Dayton Area Graduate Studies Institute (DAGSI) fellowship program for a project titled “Assessment of Team Dynamics Using Adaptive Modeling of Biometric Data.” The authors wish to thank their DAGSI sponsor Dr. Gregory Funke for his continued guidance and support throughout the project.
- 3.Jin, L., et al.: Driver cognitive distraction detection using driving performance measures. In: Discrete Dynamics in Nature and Society (2012)Google Scholar
- 5.Putze, F., Jarvis, J., Schultz, T.: Multimodal recognition of cognitive workload for multitasking in the car. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 3748–3751. IEEE Press, New York (2010)Google Scholar
- 7.Solovey, E., et al.: Classifying driver workload using physiological and driving performance data: two field studies. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2014, pp. 4057–4066. ACM, New York (2014)Google Scholar
- 8.Berka, C., et al.: EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat. Space Environ. Med. 78(5), B231–B234 (2007)Google Scholar
- 13.Elkin, C., Nittala, S., Devabhaktuni, V.: Fundamental cognitive workload assessment: a machine learning comparative approach. In: 8th International Conference on Applied Human Factors and Ergonomics (AHFE), pp. 275–284. Springer, Cham (2017)Google Scholar
- 14.Elkin, C., Devabhaktuni, V.: Analysis of alternatives for neural network training techniques in assessing cognitive workload. In: 9th International Conference on Applied Human Factors and Ergonomics (AHFE 2018), pp. 27–37. Springer, Cham (2018)Google Scholar
- 16.Natarajan, A., Xu, K.S., Eriksson, B.: Detecting divisions of the autonomic nervous system using wearables. In: IEEE 38th Annual International Conference of the Engineering in Medicine and Biology Society (EMBC), pp. 5761–5764. IEEE Press, New York (2016)Google Scholar
- 17.Deap: a dataset for emotion analysis using eeg, physiological and video signals. http://www.eecs.qmul.ac.uk/mmv/datasets/deap/
- 18.UCI machine learning repository, University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml
- 19.Nittal, S., et al.: Pilot skill level and workload prediction for sliding-scale autonomy. In: 17th IEEE International Conference on Machine Learning Applications (ICMLA), pp. 1166–1173. IEEE Press, New York (2018)Google Scholar