Empirical Comparison of Task Completion Time between Mobile Phone Models with Matched Interaction Sequences
CogTool is a predictive evaluation tool for user interfaces. We wanted to apply CogTool to an evaluation of two mobile phones, but, at the time of writing, CogTool lacks the necessary (modeling baseline) observed human performance data to allow it to make accurate predictions about mobile phone use. To address this problem, we needed to collect performance data from both novice users’ and expert users’ interactions to plug into CogTool. Whilst novice users for a phone are easy to recruit, in order to obtain observed data on expert users’ performance, we had to recruit owners of our two target mobile phone models as participants. Unfortunately, it proved to be hard to find enough owners of each target phone model. Therefore we asked if multiple similar models that had matched interaction sequences could be treated as the same model from the point of view of expert performance characteristics. In this paper, we report an empirical experimental exercise to answer this question. We compared identical target task completion time for experts across two groups of similar models. Because we found significant differences in some of the task completion times within one group of models, we would argue that it is not generally advisable to consider multiple phone models as equivalent for the purpose of obtaining observed data for predictive modeling.
KeywordsCognitive Model CogTool Evaluation Human Centered Design Human Interface Mobile Phone Systematization Usability Test
Unable to display preview. Download preview PDF.
- 2.Bellotti, V., Fukuzumi, S., Asahi, T., Suzuki, S.: User-Centered Design and Evaluation - The Big Picture. In: Proceedings of Human Computer Interaction International. Springer, Heidelberg (to appear, 2009)Google Scholar
- 5.Teo, L., John, B.E.: Comparisons of Keystroke-Level Model predictions to observed data. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2006, pp. 1421–1426. ACM, New York (2006)Google Scholar
- 8.Newell, A., Rosenbloom, P.S.: Mechanisms of skill acquisition and the law of practice. In: Rosenbloom, P.S., Laird, J.E., Newell, A. (eds.) The Soar papers. Research on integrated intelligence, vol. 1, pp. 81–135. MIT Press, Cambridge (1993)Google Scholar