Multiple Factors Mental Load Evaluation on Smartphone User Interface

  • Meng LiEmail author
  • Armagan Albayrak
  • Yu Zhang
  • Daan van Eijk
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 827)


Smartphone is nowadays the most prevalent computer system, thus a lot of attention from academia and industries has been put to evaluate its quality of use. However, Smartphone has more complex interaction modes and usage scenarios than PC and laptop. And therefore assessing its quality using a conventional usability evaluation is not sufficient. Meanwhile, the mental load serves as an acknowledged index of effort that operators have put in human-machine interaction, especially under high-demanding context. Mental load contains a set of parameters in multiple dimensions, such as primitive task performance, biological measurement(s) and subjective mental load scale, which assesses the efforts of tasks under a particular environment and operating conditions. Thus, it is suitable for evaluating complex mental work, and may indicate the use of Smartphones.

The aim of this paper is to apply a multi-dimensional method to assess the mental load of users, and find out which measurement(s) is the most suitable one to evaluate the efforts for using a smartphone. During this study, the effort on conducting tasks with four difficulty levels were assessed using measurements in three dimensions, which were (1) user performance (task accomplishment and secondary task), (2) subjective rating (NASA-TLX scale) and (3) physiological function (EDA). The values of these measurements were compared across novice, average and skilled users. The results show that: task duration and number of usability error are significantly related with mental load and change with the difficulty level of tasks; in subjective rating, Mental Demand, Effort and Frustration were highly related with mental load.


Mental load evaluation Usability Smartphone 


  1. 1.
    ISO (2001) ISO/IEC 9126-1:2001 Software engineering – Product quality – Part 1: Quality model. International Standard. International Organization for Standardization, SwitzerlandGoogle Scholar
  2. 2.
    ISO: ISO9241-11(1998) 1998 Ergonomic requirements for office work with visual display terminals (VDT’s) – Part 11: Guidance on usability. International Standard, International Organization for Standardization, SwitzerlandGoogle Scholar
  3. 3.
    Li LS (1999) Action theory and cognitive psychology in industrial design: User models and user interfaces. Dissertation, Art University of Braunschweig, BraunschweigGoogle Scholar
  4. 4.
    Kantowitz BH (1987) Mental Workload. In: Hancock PA. (ed) Advances in Psychology, vol 47, pp 81–121Google Scholar
  5. 5.
    Liao JQ (1995) Mental workload and its measurement. J Syst Eng 10(3):119–123Google Scholar
  6. 6.
    Kang WY, Yuan XG, Liu ZQ, Liu W (2008) Synthetic Evaluation method of mental workload on visual display interface in airplane cockpit. Space Med Med Eng 21(2):103–107Google Scholar
  7. 7.
    Li L, Yuan M (2011) Influential factors analysis of drivers’ mental workload with the use of vehicle navigation system. J Saf Environ 11(6):202–204Google Scholar
  8. 8.
    Cui K, Sun LY, Feng TW, Xing X (2008) New developments in measurement methodologies of mental workload. Industr Eng J 11(5):1–5Google Scholar
  9. 9.
    Cooper GE, Harper RP, Jr (1969) The Use of Pilot Rating in the Evaluation of Aircraft Handling Qualities. Report No NASA TN-D-5153. Technical Report, Ames Research Center, National Aeronautics and Space Administration. Moffett FieldGoogle Scholar
  10. 10.
    Hart SG (2006) NASA-task load index (NASA-TLX); 20 years later. In: Proceedings of the human factors and ergonomics society 50th annual meeting, vol 50. Sage Publications, Los Angeles, pp 904–908Google Scholar
  11. 11.
    Reid GB, Nygren TE (1988) The subjective workload assessment technique: a scaling procedure for measuring mental workload. Adv Psychol 52:185–218 Elsevier Science Publishers, North HollandCrossRefGoogle Scholar
  12. 12.
    Boles DB, Bursk JH, Phillips JB, Perdelwitz JR (2007) Predicting dual-task performance with the multiple resources questionnaire (MRQ). Hum Factors 49(1):32–45CrossRefGoogle Scholar
  13. 13.
    Galy E, Cariou M, Mélan C (2011) What is the relationship between mental workload factors and cognitive load types? Int J Psychophysiol 83(3):269–275CrossRefGoogle Scholar
  14. 14.
    Shingledecker CA, Crabtree MS, Simons JC et al (1980) Subsidiary Radio Communications Tasks for Workload Assessment in R&D Simulations I. Task Development and Workload Scaling. Technical Report, Systems Research Labs Inc, Dayton OhioGoogle Scholar
  15. 15.
    Horst RL, Johnson R, Donchin E (1980) Event-related brain potentials and subjective probability in a learning task. Mem Cognit 8(5):476–488CrossRefGoogle Scholar
  16. 16.
    Ahlstrom U, Friedman-Berg FJ (2006) Using eye movement activity as a correlate of cognitive workload. Int J Industr Ergon 36(7):623–636CrossRefGoogle Scholar
  17. 17.
    Gunn CG, Wolf S, Block RT et al (1972) Psychophysiology of the cardiovascular system. In: Greenfield NS, Sternbach RA (eds) Handbook of psychophysiology. Holt, Rinehart & Winston, New York, pp 457–483Google Scholar
  18. 18.
    Suzuki S, Kumano H, Sakano Y (2003) Effects of effort and distress coping processes on psychophysiological and psychological stress responses. Int J Psychophysiol 47(2):117–128CrossRefGoogle Scholar
  19. 19.
    Reinhardt T, Schmahl C, Wüst S, Bohus M (2012) Salivary cortisol, heart rate, electrodermal activity and subjective stress responses to the mannheim multicomponent stress test (MMST). Psychiatry Res 198(1):106–111CrossRefGoogle Scholar
  20. 20.
    Moya-Albiol L, Sanchis-Calatayud MV, Sariñana-González P, De Andrés-García S, Romero-Martínez Á, González-Bono E (2012) P03-425 - Electrodermal activity in response to a set of mental tasks in caregivers of persons with autism spectrum disorders. Eur Psychiatry 26(1):1595Google Scholar
  21. 21.
    Affectiva (2012) Liberate yourself from the lab: Q Sensor measures EDA in the wild. Affectiva QTM Solutions White PaperGoogle Scholar
  22. 22.
    Wang J, Fang WN, Li GY (2010) Mental workload evaluation method based on multi-resource theory model. J. Beijing Jiaotong Univ. 34(6):107–110Google Scholar
  23. 23.
    Peng XW, He QC, Ji T, Wang ZL, Yang L (2006) Mental workload for mental arithmetic on visual display terminal. Chin J Industr Hyg Occup Dis 24(12):726–729Google Scholar
  24. 24.
    Li JB, Xu BH (2009) synthetic assessment of cognitive load in human-machine interaction process. Acta Psychologica Sinica 41(1):35–43CrossRefGoogle Scholar
  25. 25.
    Yu YH, Li ZJ (2011) Study of sonically enhanced menu interaction for mobile terminals. Appl Res Comput 28(10):3742–3745Google Scholar
  26. 26.
    Jimenez-Molina A, Retamal C, Lira H (2018) Using psychophysiological sensors to assess mental workload during web browsing. Sensors 18(2):458CrossRefGoogle Scholar
  27. 27.
    Li M (2008) Comparison of Usability Evaluation Method Based-on Needs of Software Development. Master Thesis, Xi’an Jiaotong University, Xi’anGoogle Scholar
  28. 28.
    O’Donnell RD, Eggemeier FT (1986) Workload assessment methodology. In: Boff KR, Kaufman L, Thomas JP (eds) Handbook of perception and human performance, vol II. Wiley, New York, pp 42–43Google Scholar
  29. 29.
    Galy E, Cariou M, Mélan C (2012) What is the relationship between mental workload factors and cognitive load types? Int J Psychophysiol 83(3):269–275CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Delft University of TechnologyDelftNetherlands
  2. 2.Xi’an Jiaotong UniversityXi’anChina

Personalised recommendations