Cognitive Ergonomic Evaluation Metrics and Methodology for Interactive Information System

  • Yu Zhang
  • Jianhua Sun
  • Ting JiangEmail author
  • Zengyao Yang
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 965)


In the face of complex information interactive system, it is essential to evaluate products achieve system performance within users cognitive capacity. Most of the research about ergonomic evaluation mainly focus on the macro ergonomic method, which not focus on concrete design problem at the micro level. This paper focuses on how to identify and predict cognitive ergonomic problems based user action and cognitive model and establishes the mapping relationship between cognitive ergonomic problems and real-time continuous measured data in order to let the evaluation results play a direct role in the design. The methodology was applied to evaluate the ergonomic quality of IETM used by astronauts in the space station, which including make flight plans, do experiments, in-orbit maintenance, and so on. A series of standardized evaluation procedures were designed to explore the possibility of remote ergonomic measurement for long-term orbiting operation.


Cognitive ergonomics Evaluation Quantitative analysis Eye-tracking Interactive system 



This research has been supported by the Open Funding Project of National Key Laboratory of Human Factors Engineering, Grant NO. SYFD170051809 K.


  1. 1.
    What is ergonomics? International Ergon Associates (IEA) (2012).
  2. 2.
    Hollnagel, E.: Cognitive ergonomics: it’s all in the mind. Ergonomics 40(10), 1170–1182 (1997)CrossRefGoogle Scholar
  3. 3.
    Berlin, C., Adams, C.: Production Ergonomics: Designing Work Systems to Support Optimal Human Performance, pp. 83–106. Ubiquity Press, London (2017)Google Scholar
  4. 4.
    ISO/IEC 9126. Information technology - software product evaluation – quality characteristics and guidance for their use. ISO/IEC (1991)Google Scholar
  5. 5.
    ISO 9241. Ergonomic requirements for office work with visual display terminals, Part 8. Requirements for displayed colors. ISO (1994)Google Scholar
  6. 6.
    Nielsen, J., Mack, R.L.: Heuristic evaluation. In: Usability Inspection Methods. Wiley, New York (2010)Google Scholar
  7. 7.
    Shackel, B.: Usability- context, framework, definition, design and evaluation. Interact. Comput. 21(5–6), 339–346 (2009)CrossRefGoogle Scholar
  8. 8.
    Lewis, C., Polson, P., Wharton, C., et al.: Testing a walkthrough methodology for theory-based design of walk –up-and-use interfaces. In: CHI’90 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 235–242. ACM, New York (1990)Google Scholar
  9. 9.
    Longo, L.: Experienced mental workload, perception of usability, their interaction and impact on task performance. PLoS ONE 13(8), e0199661 (2018). Scholar
  10. 10.
    ISO 10075-1:2017. Ergonomic principles related to mental workload – Part 1: general issues and concepts, terms and definitions. ISO (2017)Google Scholar
  11. 11.
    Young, M., Brookhuis, K., Wickens, C., Hancock, P.: State of science: mental workload in ergonomics. Ergonomics 58(1), 1–17 (2015)CrossRefGoogle Scholar
  12. 12.
    Hancock, P.: Whither workload? Mapping a path for its future development. In: International Symposium on Human Mental Workload: Models and Applications, pp. 3–17. Springer (2017)Google Scholar
  13. 13.
    Wickens, C.: Mental workload: assessment, prediction and consequences. In: International Symposium on Human Mental Workload: Models and Applications, pp. 18–29. Springer (2017)Google Scholar
  14. 14.
    Ariza, F., Kalra, D., Potts, H.W.: How do clinical information systems affect the cognitive demands of general practitioners? Usability study with a focus on cognitive workload. J. Innov. Health Inform. 22(4), 379–390 (2015)CrossRefGoogle Scholar
  15. 15.
    Cain, B.: A review of the mental workload literature. In: Defence Research & Development Canada, Human System Integration (2007)Google Scholar
  16. 16.
    Rubio, S., Diaz, E., Martin, J., Puente, J.M.: Evaluation of subjective mental workload: a comparison of SWAT, NASA-TLX, and workload profile methods. Appl. Psychol. 53(1), 61–86 (2004)CrossRefGoogle Scholar
  17. 17.
    Hart, S.G.: NASA-task load index (NASA-TLX); 20 years later. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 50(9), 904–908 (2006)CrossRefGoogle Scholar
  18. 18.
    Reid, G.B., Nygren, T.E.: The subjective workload assessment technique: a scaling procedure for measuring mental workload. Adv. Psychol. 52, 185–218 (1988)CrossRefGoogle Scholar
  19. 19.
    Boles, D.B., Bursk, J.H., Phillips, J.B., Perdelwitz, J.R.: Predicting dual-task performance with The Multiple Resources Questionnaire (MRQ). Hum. Factors 49, 32–45 (2007)CrossRefGoogle Scholar
  20. 20.
    Eggemeier, T., O’Donnell, R.: A conceptual framework for development of a workload assessment methodology. In: Defense Technical Information Center OAI-PMH Repository (United States) (1998)Google Scholar
  21. 21.
    Charles, R.L., Nixon, J.: Measuring mental workload using physiological measures: a systematic review. Appl. Ergonomics 74, 221–232 (2019)CrossRefGoogle Scholar
  22. 22.
    O’Donnell, C.R.D., Eggemeier, F.T.: Workload assessment methodology. In: Measurement Technique, Ch42, pp. 42-5 (1986)Google Scholar
  23. 23.
    Dominique, L.S., Christian Bastien, J.M.: Ergonomic criteria for evaluating the ergonomic quality of interactive systems. Behav. Inf. Technol. 16(4–5), 220–231 (1997)Google Scholar
  24. 24.
    Neville, A.S., Mark, S. Y., Catherine, H.: Guide to Methodology in Ergonomics Designing for Human Use, pp. 9–76. Taylor & Francis, London (2014) Google Scholar
  25. 25.
    Romaric, M., Andre, W.K., Marie-Catherine, B.Z., Elizabeth, M.B.: Insights and limits of usability evaluation methods along the health information technology lifecycle. Stud. Health Technol. Inform. 210, 115–119. EEMI (2015)Google Scholar
  26. 26.
    Hornbaek, K.: Current practice in measuring usability: challenges to usability studies and research. Int. J. Hum. Comput. Stud. 64(2), 79–102 (2006)CrossRefGoogle Scholar
  27. 27.
    Alper, A., Duygun, E.B., et al.: Evaluation of a surgical interface for robotic cryoablation task using an eye-tracking system. Int. J. Hum. Comput. Stud. 95, 39–53 (2016)CrossRefGoogle Scholar
  28. 28.
    Imbert, J.p., Hodgetts, H.M., Parise, R., Vachon, F., Dehais, F., Tremblay, S.: Attentional costs and failures in air traffic control notifications. Ergonomics 57, 1817–1832. (2014)CrossRefGoogle Scholar
  29. 29.
    Smith, P.A.: Towards a practical measure of hypertext usability. Interact. Comput. 8, 365–381 (1996)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Yu Zhang
    • 1
  • Jianhua Sun
    • 1
  • Ting Jiang
    • 2
    Email author
  • Zengyao Yang
    • 1
  1. 1.Department of Industrial Design, School of Mechanical EngineeringXi’an Jiaotong UniversityXi’anChina
  2. 2.National Key Laboratory of Human Factors EngineeringChina Astronauts Research and Training CenterBeijingChina

Personalised recommendations