Advertisement

Comparative Analysis of Machine Learning Techniques in Assessing Cognitive Workload

  • Colin ElkinEmail author
  • Vijay Devabhaktuni
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 953)

Abstract

Cognitive workload refers to the amount of mental capacity that is stored in working memory and remains a vital yet challenging aspect of many human factors applications. This paper presents a comprehensive analysis of alternatives for different machine learning techniques used for assessing mental workload. This methodology consists of five techniques and four datasets. Each dataset consists of physiological factors such as electrodermal activity (EDA) or heart rate data as well as subjective factors. When evaluating our techniques, we compare accuracy, runtime, and F1 score across multiple different method-specific experimental parameters. Ultimately, the results indicate that ANNs perform best on datasets with large numbers of inputs and classification options, while decision trees are most ideal under a lower number of output possibilities. By understanding these individual strengths and weaknesses, we can ultimately improve the balance of ideal human performance through early detection of cognitive overload or underload.

Keywords

Cognitive workload Machine learning Artificial neural networks Support vector machines K-nearest neighbors Decision trees Random forests 

Notes

Acknowledgments

This research is supported by the Dayton Area Graduate Studies Institute (DAGSI) fellowship program for a project titled “Assessment of Team Dynamics Using Adaptive Modeling of Biometric Data.” The authors wish to thank their DAGSI sponsor Dr. Gregory Funke for his continued guidance and support throughout the project.

References

  1. 1.
    Wilson, G.F., Russell, C.A.: Real-time assessment of mental workload using psychophysiological measures and artificial neural networks. Hum. Factors 45(4), 635–644 (2016)CrossRefGoogle Scholar
  2. 2.
    Baldwin, C.L., Penaranda, B.N.: Adaptive training using an artificial neural network and EEG metrics for within- and cross-task workload classification. NeuroImage 59(1), 48–56 (2012)CrossRefGoogle Scholar
  3. 3.
    Jin, L., et al.: Driver cognitive distraction detection using driving performance measures. In: Discrete Dynamics in Nature and Society (2012)Google Scholar
  4. 4.
    Son, J., Oh, H., Park, M.: Identification of driver cognitive workload using support vector machines with driving performance, physiology and eye movement in a driving simulator. Int. J. Precis. Eng. Manuf. 14(8), 1321–1327 (2013)CrossRefGoogle Scholar
  5. 5.
    Putze, F., Jarvis, J., Schultz, T.: Multimodal recognition of cognitive workload for multitasking in the car. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 3748–3751. IEEE Press, New York (2010)Google Scholar
  6. 6.
    Liang, Y., Reyes, M., Lee, J.: Real-time detection of driver cognitive distraction using support vector machines. IEEE Trans. Intell. Transp. Syst. 8(2), 340–350 (2007)CrossRefGoogle Scholar
  7. 7.
    Solovey, E., et al.: Classifying driver workload using physiological and driving performance data: two field studies. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2014, pp. 4057–4066. ACM, New York (2014)Google Scholar
  8. 8.
    Berka, C., et al.: EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat. Space Environ. Med. 78(5), B231–B234 (2007)Google Scholar
  9. 9.
    Niaz, M., Logie, R.H.: Working memory, mental capacity and science education: towards an understanding of the ‘working memory overload hypothesis’. Oxford Rev. Educ. 19(4), 511–525 (1993)CrossRefGoogle Scholar
  10. 10.
    Lenneman, J., Backs, R.: Cardiac autonomic control during simulated driving with a concurrent verbal working memory task. Hum. Factors 51(3), 404–418 (2009)CrossRefGoogle Scholar
  11. 11.
    Reimer, B., Mehler, B.: The impact of cognitive workload on physiological arousal in young adult drivers: a field study and simulation validation. Ergonomics 54(10), 932–942 (2011)CrossRefGoogle Scholar
  12. 12.
    Collet, C., et al.: Physiological and behavioural changes associated to the management of secondary tasks while driving. Appl. Ergon. 40(6), 1041–1046 (2009)CrossRefGoogle Scholar
  13. 13.
    Elkin, C., Nittala, S., Devabhaktuni, V.: Fundamental cognitive workload assessment: a machine learning comparative approach. In: 8th International Conference on Applied Human Factors and Ergonomics (AHFE), pp. 275–284. Springer, Cham (2017)Google Scholar
  14. 14.
    Elkin, C., Devabhaktuni, V.: Analysis of alternatives for neural network training techniques in assessing cognitive workload. In: 9th International Conference on Applied Human Factors and Ergonomics (AHFE 2018), pp. 27–37. Springer, Cham (2018)Google Scholar
  15. 15.
    Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Natarajan, A., Xu, K.S., Eriksson, B.: Detecting divisions of the autonomic nervous system using wearables. In: IEEE 38th Annual International Conference of the Engineering in Medicine and Biology Society (EMBC), pp. 5761–5764. IEEE Press, New York (2016)Google Scholar
  17. 17.
    Deap: a dataset for emotion analysis using eeg, physiological and video signals. http://www.eecs.qmul.ac.uk/mmv/datasets/deap/
  18. 18.
    UCI machine learning repository, University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml
  19. 19.
    Nittal, S., et al.: Pilot skill level and workload prediction for sliding-scale autonomy. In: 17th IEEE International Conference on Machine Learning Applications (ICMLA), pp. 1166–1173. IEEE Press, New York (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.ECE DepartmentPurdue University NorthwestHammondUSA

Personalised recommendations