Abstract
Different questionnaires assessing the usability of two multimodal systems and one unimodal system were compared. Each participant (N=21) performed several tasks with each device and was afterwards asked to rate the system by filling out different questionnaires. The results show that standardized questionnaires are applicable only to a limited extent. Despite some concordance, the results differ considerably and thus indicate the need for the development of a valid and reliable questionnaire covering the usability and quality of multimodal systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bernsen, N.O., Dybkjær, L.: Evaluation of Spoken Multimodal Conversation. In: Proceedings of the Sixth International Conference on Multimodal Interfaces (ICMI 2004), Penn State University, USA, pp. 38–45 (2004)
Brooke, J.: SUS: A ‘quick and dirty’ usability scale. In: Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, I.L. (eds.) Usability Evaluation in Industry, pp. 189–194. Taylor & Francis, London (1996)
Hassenzahl, M., Burmester, M., Koller, F.: AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In: Ziegler, J., Szwillus, G. (eds.) Mensch & Computer 2003. Interaktion in Bewegung, pp. 187–196. B.G. Teubner, Stuttgart (2003)
Hone, K.S., Graham, R.: Towards a tool for the subjective assessment of speech system interfaces (SASSI). Natural Language Engineering 6(3/4), 287–305 (2000)
Kirakowski, J., Corbett, M.: SUMI: The software usability measurement inventory. British Journal of Educational Technology 24(3), 210–212 (1993)
Larsen, L.B.: Assessment of spoken dialogue system usability - what are we really measuring? In: Proceedings of Eurospeech 2003, pp. 1945–1948 (2003)
Petersen, M.G.: Towards Usability Evaluation of Multimedia Applications. Crossroads 4(4), 3–7 (1998)
Porteous, M., Jurek, K., Corbett, M.: SUMI: User Handbook. Human Factors Research Group, University of Cork, Ireland (1993)
Sturm, J.: On the usability of multimodal interaction for mobile access to information services. PhD thesis, Radboud University Nijmegen, Nijmegen, The Netherlands (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wechsung, I., Naumann, A.B. (2008). Evaluation Methods for Multimodal Systems: A Comparison of Standardized Usability Questionnaires. In: André, E., Dybkjær, L., Minker, W., Neumann, H., Pieraccini, R., Weber, M. (eds) Perception in Multimodal Dialogue Systems. PIT 2008. Lecture Notes in Computer Science(), vol 5078. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69369-7_32
Download citation
DOI: https://doi.org/10.1007/978-3-540-69369-7_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69368-0
Online ISBN: 978-3-540-69369-7
eBook Packages: Computer ScienceComputer Science (R0)