Abstract
The usability now serves as a fundamental quality of a computational device, e.g. smartphone. Moreover, the smartphone has firmly embedded into our daily life as an indispensable part, so the context and style that user may interact with them are largely different from a decade ago. Nowadays, testing usability with end user has become a common sense. Thus, how valid a usability evaluation method could assess the ‘extent to which a product can be used by specified users’ (ISO 9241-11) to facilitate software design becomes an interesting question to explore.
In this research, three usability evaluation methods are compared. Among these methods, IsoMetrics is a standard questionnaire aiming at offer usability data for summative and formative evaluation; SUMI aims to assess quality of software product from end users perspective; User Model Checklist is a method based on user’s cognition-motor chain in specific tasks. The coverage and amount of usability issues, user’s effort of evaluation and software developer’s feedback on evaluation result are compared under a simulated usability test on SMS function with a smartphone. The result indicate that User Model Checklist could cover 90.4% of the usability issues found by IsoMetrics and SUMI, while 26.3% usability issues found by User Model Checklist could not be covered by IsoMetrics and SUMI. Users put highest effort on accomplish IsoMetrics and lowest effort on User Model Checklist. Moreover, the feedbacks from the developers show that the User Model Checklist requires lower usability knowledge, offers clearer improvement points and supports detailed design better.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Macleod M, Bowden R, Bevan N, Curson I (1997) The MUSiC performance measurement method. Behav Inf Technol 16(4–5):279–293
Nokia Corporation (2004) Series 60 Developer Platform 2.0: Usability Guidelines For Enterprise Applications. http://www.forum.nokia.com/usability
Weiss S (2003) Handheld usability. WileySons, Chichester
Nielsen J (2012) Usability 101: Introduction to Usability. https://www.nngroup.com/articles/usability-101-introduction-to-usability
Nielsen J (1994) Usability engineering. Elsevier, London
Lathan CE, Newman DJ, Sebrechts MM, Doarn CR (1997) Evaluating a web-based interface for internet telemedicine. NASA, Washington
Apple Inc. (2018) Human Interface Guideline. https://developer.apple.com/ios/human-interface-guidelines/overview/themes/
Hornbæk K (2006) Current practice in measuring usability: Challenges to usability studies and research. Int J Hum Comput Stud 64(2):79–102
Hartson HR, Andre TS, Williges RC (2003) Criteria for evaluating usability evaluation methods. Int J Hum Comput Interact 15(1):145–181
Li LS (2007) Design investigation. China Architecture & Building Press, Beijing
International Organization for Standardization (2002) ISO/TR 16982:2002(E) Ergonomics of human-system interaction—Usability method supporting human-centred design. ISO, Switzerland
Gediga G, Hamborg KC, Düntsch I (1999) The IsoMetrics usability inventory: an operationalization of ISO 9241-10 supporting summative and formative evaluation of software systems. Behav Inf Technol 18(3):151–164
Kirakowski J, Corbett M (1993) SUMI: the software usability measurement inventory. Br J Educ Technol 24(3):210–212
Follmer S, Leithinger D, Olwal A, Hogge A, Ishii H (2013) inFORM: dynamic physical affordances and constraints through shape and object actuation. In: UIST, vol 13
Solso RL, Johnson HH (1989) An introduction to experimental design in psychology: a case approach, 3rd edn. Harper & Row Publishers Inc., New York
Rasmussen J (1983) Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE Trans Syst Man Cybern 3:257–266
International Organization for Standardization (2006) ISO 9241-110:2006 Ergonomics of human-system interaction – Part 110: Dialogue principles. ISO, Switzerland
Li LS (2004) Human computer interface design. Science Press, Beijing
Berenguer A, Goncalves J, Hosio S, Ferreira D, Anagnostopoulos T, Kostakos V (2017) Are Smartphones Ubiquitous?: an in-depth survey of smartphone adoption by seniors. IEEE Consum Electron Mag 6(1):104–110
Lee H, Ahn H, Nguyen TG, Choi SW, Kim DJ (2017) Comparing the self-report and measured smartphone usage of college students: a pilot study. Psychiatry Invest 14(2):198–204
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, M., Albayrak, A., Zhang, Y., van Eijk, D., Yang, Z. (2019). Comparison of Questionnaire Based and User Model Based Usability Evaluation Methods. In: Bagnara, S., Tartaglia, R., Albolino, S., Alexander, T., Fujita, Y. (eds) Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018). IEA 2018. Advances in Intelligent Systems and Computing, vol 824. Springer, Cham. https://doi.org/10.1007/978-3-319-96071-5_110
Download citation
DOI: https://doi.org/10.1007/978-3-319-96071-5_110
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-96070-8
Online ISBN: 978-3-319-96071-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)