Comparison of Questionnaire Based and User Model Based Usability Evaluation Methods
The usability now serves as a fundamental quality of a computational device, e.g. smartphone. Moreover, the smartphone has firmly embedded into our daily life as an indispensable part, so the context and style that user may interact with them are largely different from a decade ago. Nowadays, testing usability with end user has become a common sense. Thus, how valid a usability evaluation method could assess the ‘extent to which a product can be used by specified users’ (ISO 9241-11) to facilitate software design becomes an interesting question to explore.
In this research, three usability evaluation methods are compared. Among these methods, IsoMetrics is a standard questionnaire aiming at offer usability data for summative and formative evaluation; SUMI aims to assess quality of software product from end users perspective; User Model Checklist is a method based on user’s cognition-motor chain in specific tasks. The coverage and amount of usability issues, user’s effort of evaluation and software developer’s feedback on evaluation result are compared under a simulated usability test on SMS function with a smartphone. The result indicate that User Model Checklist could cover 90.4% of the usability issues found by IsoMetrics and SUMI, while 26.3% usability issues found by User Model Checklist could not be covered by IsoMetrics and SUMI. Users put highest effort on accomplish IsoMetrics and lowest effort on User Model Checklist. Moreover, the feedbacks from the developers show that the User Model Checklist requires lower usability knowledge, offers clearer improvement points and supports detailed design better.
KeywordsUsability evaluation comparison IsoMetrics SUMI User Model Checklist
- 2.Nokia Corporation (2004) Series 60 Developer Platform 2.0: Usability Guidelines For Enterprise Applications. http://www.forum.nokia.com/usability
- 3.Weiss S (2003) Handheld usability. WileySons, ChichesterGoogle Scholar
- 4.Nielsen J (2012) Usability 101: Introduction to Usability. https://www.nngroup.com/articles/usability-101-introduction-to-usability
- 6.Lathan CE, Newman DJ, Sebrechts MM, Doarn CR (1997) Evaluating a web-based interface for internet telemedicine. NASA, WashingtonGoogle Scholar
- 7.Apple Inc. (2018) Human Interface Guideline. https://developer.apple.com/ios/human-interface-guidelines/overview/themes/
- 10.Li LS (2007) Design investigation. China Architecture & Building Press, BeijingGoogle Scholar
- 11.International Organization for Standardization (2002) ISO/TR 16982:2002(E) Ergonomics of human-system interaction—Usability method supporting human-centred design. ISO, SwitzerlandGoogle Scholar
- 14.Follmer S, Leithinger D, Olwal A, Hogge A, Ishii H (2013) inFORM: dynamic physical affordances and constraints through shape and object actuation. In: UIST, vol 13Google Scholar
- 15.Solso RL, Johnson HH (1989) An introduction to experimental design in psychology: a case approach, 3rd edn. Harper & Row Publishers Inc., New YorkGoogle Scholar
- 17.International Organization for Standardization (2006) ISO 9241-110:2006 Ergonomics of human-system interaction – Part 110: Dialogue principles. ISO, SwitzerlandGoogle Scholar
- 18.Li LS (2004) Human computer interface design. Science Press, BeijingGoogle Scholar