Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Human Mental Workload: Models and Applications

H-WORKLOAD 2018: Human Mental Workload: Models and Applications pp 13–22Cite as

Assessing Workload in Human-Machine Teams from Psychophysiological Data with Sparse Ground Truth

  • Conference paper
  • First Online:

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1012))

Abstract

Data-driven approaches to human workload assessment generally attempt to induce models from a collection of available data and a corresponding ground truth comprising self-reported measures of actual workload. However, it is often not feasible to elicit self-assessed workload ratings with great frequency. As part of an ongoing effort to improve the effectiveness of human-machine teams through real-time human workload monitoring, we explore the utility of transfer learning in situations where there is sparse subject-specific ground truth from which to develop accurate predictive models of workload. Our approach induces a workload model from the psychophysiological data collected from subjects operating a remotely piloted aircraft simulation program. Psychophysiological measures were collected from wearable sensors, and workload was self-assessed using the NASA Task Load Index. Our results provide evidence that models learned from psychophysiological data collected from other subjects outperform models trained on a limited amount of data for a given subject.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Hancock, P.A.: Whither workload? Mapping a path for its future development. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2017. CCIS, vol. 726, pp. 3–17. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61061-0_1

    Chapter  Google Scholar 

  2. Longo, L.: Mental workload in medicine: foundations, applications, open problems, challenges and future perspectives. In: 2016 IEEE 29th International Symposium on Computer-Based Medical Systems (CBMS), pp. 106–111 (2016). https://doi.org/10.1109/cbms.2016.36

  3. Wickens, C.D.: Mental workload: assessment, prediction and consequences. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2017. CCIS, vol. 726, pp. 18–29. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61061-0_2

    Chapter  Google Scholar 

  4. Balfe, N., Crowley, K., Smith, B., Longo, L.: Estimation of train driver workload: extracting taskload measures from on-train-data-recorders. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2017. CCIS, vol. 726, pp. 106–119. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61061-0_7

    Chapter  Google Scholar 

  5. Chen, J.Y., Haas, E.C., Barnes, M.J.: Human performance issues and user interface design for teleoperated robots. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 37(6), 1231–1245 (2007). https://doi.org/10.1109/tsmcc.2007.905819

    Article  Google Scholar 

  6. Edwards, T., Martin, L., Bienert, N., Mercer, J.: The relationship between workload and performance in air traffic control: exploring the influence of levels of automation and variation in task demand. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2017. CCIS, vol. 726, pp. 120–139. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61061-0_8

    Chapter  Google Scholar 

  7. Fan, J., Smith, A.P.: The impact of workload and fatigue on performance. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2017. CCIS, vol. 726, pp. 90–105. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61061-0_6

    Chapter  Google Scholar 

  8. Hancock, P.A., Williams, G., Manning, C.M.: Influence of task demand characteristics on workload and performance. Int. J. Aviat. Psychol. 5(1), 63–86 (1995). https://doi.org/10.1207/s15327108ijap0501_5

    Article  Google Scholar 

  9. Longo, L.: Experienced mental workload, perception of usability, their interaction and impact on task performance. PLoS ONE 13(8), e0199661 (2018). https://doi.org/10.1371/journal.pone.0199661

    Article  Google Scholar 

  10. Orasanu, J.M.: Shared problem models and flight crew performance. In: Johnston, N., McDonald, N., Fuller, R. (eds.) Aviation Psychology in Practice. Ashgate Publishing Group, Aldershot (1994). https://doi.org/10.4324/9781351218825

    Book  Google Scholar 

  11. Smith, A.P., Smith, H.N.: Workload, fatigue and performance in the rail industry. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2017. CCIS, vol. 726, pp. 251–263. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61061-0_17

    Chapter  Google Scholar 

  12. Tong, S., Helman, S., Balfe, N., Fowler, C., Delmonte, E., Hutchins, R.: Workload differences between on-road and off-road manoeuvres for motorcyclists. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2017. CCIS, vol. 726, pp. 239–250. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61061-0_16

    Chapter  Google Scholar 

  13. Moustafa, K., Luz, S., Longo, L.: Assessment of mental workload: a comparison of machine learning methods and subjective assessment techniques. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2017. CCIS, vol. 726, pp. 30–50. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61061-0_3

    Chapter  Google Scholar 

  14. Kantowitz, B.H.: Mental workload. In: Hancock, P.A. (ed.) Advances in Psychology, vol. 47, pp. 81–121. North-Holland (1987). https://doi.org/10.1016/s0166-4115(08)62307-9

    Google Scholar 

  15. Longo, L., Barrett, S.: A computational analysis of cognitive effort. In: Nguyen, N.T., Le, M.T., Świątek, J. (eds.) ACIIDS 2010. LNCS (LNAI), vol. 5991, pp. 65–74. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12101-2_8

    Chapter  Google Scholar 

  16. Moray, N. (ed.): Mental Workload: Its Theory and Measurement, vol. 8. Springer, New York (2013). https://doi.org/10.1007/978-1-4757-0884-4

    Book  Google Scholar 

  17. Appriou, A., Cichocki, A., Lotte, F.: Towards robust neuroadaptive HCI: exploring modern machine learning methods to estimate mental workload from EEG signals. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, p. LBW615. ACM (2018). https://doi.org/10.1145/3170427.3188617

  18. Wilson, G.F., Russell, C.A.: Real-time assessment of mental workload using psychophysiological measures and artificial neural networks. Hum. Factors 45(4), 635–644 (2003). https://doi.org/10.1518/hfes.45.4.635.27088

    Article  Google Scholar 

  19. Zhang, J., Yin, Z., Wang, R.: Recognition of mental workload levels under complex human–machine collaboration by using physiological features and adaptive support vector machines. IEEE Trans. Hum.-Mach. Syst. 45(2), 200–214 (2015). https://doi.org/10.1109/thms.2014.2366914

    Article  Google Scholar 

  20. Prinzel III, L.J., Parasuraman, R., Freeman, F.G., Scerbo, M.W., Mikulka, P.J., Pope, A.T.: Three experiments examining the use of electroencephalogram, event-related potentials, and heart-rate variability for real-time human-centered adaptive automation design. Report TP-2003-212442, NASA, Langley Research Center, Hampton (2003)

    Google Scholar 

  21. Roscoe, A.H.: Assessing pilot workload. Why measure heart rate, HRV and respiration? Biol. Psychol. 34(2–3), 259–287 (1992). https://doi.org/10.1016/0301-0511(92)90018-p

    Article  Google Scholar 

  22. Veltman, J.A., Gaillard, A.W.K.: Physiological workload reactions to increasing levels of task difficulty. Ergonomics 41(5), 656–669 (1998). https://doi.org/10.1080/001401398186829

    Article  Google Scholar 

  23. Verwey, W.B., Veltman, H.A.: Detecting short periods of elevated workload: a comparison of nine workload assessment techniques. J. Exp. Psychol.: Appl. 2(3), 270 (1996). https://doi.org/10.1037/1076-898X.2.3.270

    Article  Google Scholar 

  24. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010). https://doi.org/10.1109/tkde.2009.191

    Article  Google Scholar 

  25. Wu, D., Lance, B.J., Parsons, T.D.: Collaborative filtering for brain-computer interaction using transfer learning and active class selection. PLoS ONE 8(2), e56624 (2013). https://doi.org/10.1371/journal.pone.0056624

    Article  Google Scholar 

  26. Hoepf, M., Middendorf, M., Epling, S., Galster, S.: Physiological indicators of workload in a remotely piloted aircraft simulation. In: 18th International Symposium on Aviation Psychology, pp. 428–433. Curran, Dayton (2015)

    Google Scholar 

  27. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Advances in Psychology, vol. 52, pp. 139–183. North-Holland (1988). https://doi.org/10.1016/s0166-4115(08)62386-9

    Google Scholar 

  28. Christensen, J.C., Estepp, J.R., Wilson, G.F., Russell, C.A.: The effects of day-to-day variability of physiological data on operator functional state classification. NeuroImage 59(1), 57–63 (2012). https://doi.org/10.1016/j.neuroimage.2011.07.091

    Article  Google Scholar 

  29. Le Cessie, S., Van Houwelingen, J.C.: Ridge estimators in logistic regression. Appl. Stat. 41(1), 191–201 (1992). https://doi.org/10.2307/2347628

    Article  MATH  Google Scholar 

  30. Witten, I.H., Frank, E., Hall, M.A., Pal, C.J.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, Burlington (2011). https://doi.org/10.1016/c2009-0-19715-5

    Book  Google Scholar 

  31. Youden, W.J.: Index for rating diagnostic tests. Cancer, 3(1), 32–35 (1950). https://doi.org/10.1002/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3

    Article  Google Scholar 

Download references

Acknowledgments

This material is based upon work supported by the United States Air Force under Contract No. FA8650-15-C-6669. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Air Force.

Author information

Authors and Affiliations

  1. Stottler Henke Associates, Inc., 1107 NE 45th Street, Suite 310, Seattle, WA, 98105, USA

    David Dearing, Aaron Novstrup & Terrance Goan

Authors
  1. David Dearing
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aaron Novstrup
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Terrance Goan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Dearing .

Editor information

Editors and Affiliations

  1. Dublin Institute of Technology, Dublin, Ireland

    Luca Longo

  2. Dublin Institute of Technology, Dublin, Ireland

    M. Chiara Leva

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dearing, D., Novstrup, A., Goan, T. (2019). Assessing Workload in Human-Machine Teams from Psychophysiological Data with Sparse Ground Truth. In: Longo, L., Leva, M. (eds) Human Mental Workload: Models and Applications. H-WORKLOAD 2018. Communications in Computer and Information Science, vol 1012. Springer, Cham. https://doi.org/10.1007/978-3-030-14273-5_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-14273-5_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-14272-8

  • Online ISBN: 978-3-030-14273-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation