Abstract
Users of multimodal systems have to choose between different interaction strategies. Thereby the number of interaction steps to solve a task can vary across the available modalities. In this work we introduce such a task and present empirical data that shows that strategy selection of users is affected by modality specific shortcuts. The system under investigation offered touch screen and speech as input modalities. We introduce a first version of an ACT-R model that uses the architectures-inherent mechanisms production compilation and utility learning to identify modality-specific shortcuts. A simple task analysis is implemented in declarative memory. The model reasonably accurate matches the human data. In our further work we will try to get a better fit by extending the model with further influence factors of modality selection like speech recognition errors. Further the model will be refined concerning the cognitive processes of speech production and touch screen interaction.
Chapter PDF
Similar content being viewed by others
References
Möller, S.: Assessment and Evaluation of Speech-Based Interactive Systems: From Manual Annotation to Automatic Usability Evaluation. Speech Technology, 301–322 (2010)
Baker, S., Au, F., Dobbie, G., Warren, I.: Automated Usability Testing Using HUI Analyzer. In: 19th Australian Conference on Software Engineering, ASWEC 2008, pp. 579–588 (2008)
Möller, S., Englert, R., Engelbrecht, K., Hafner, V., Jameson, A.: MeMo: Towards Automatic Usability Evaluation of Spoken Dialogue Services by User Error Simulations. System, 1–4 (2006)
John, B.E., Salvucci, D.: Multipurpose prototypes for assessing user interfaces in pervasive computing systems. Pervasive Computing 4, 27–34 (2005)
Jameson, A., Mahr, A., Kruppa, M., Rieger, A., Schleicher, R.: Looking for Unexpected Consequences of Interface Design Decisions: The MeMo Workbench. In: 19th Australian Conference on Software Engineering, ASWEC 2008, pp. 579–588 (2008)
Wickens, C.D.: Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science 3, 159–177 (2002)
McCrasken, J.H., Aldrich, T.B.: Analysis of selected LHX mission functions: Implications for operator workload and system automation goals (Technical Note ASI479-024-84). Fort Rucker, AL (1984)
Bierbaum, C.R., Szabo, S.M., Aldrich, T.B.: A comprehensive task analysis of the UH-60 mission with crew workload estimates and preliminary decision rules for developing a UH-60 workload prediction model (Technical Report ASI690-302-87[B], Vol I, II, III, IV). Fort Rucker, AL (1987)
Wechsung, I., Engelbrecht, K.-P., Naumann, A., Möller, S., Schaffer, S., Schleicher, R.: Investigating Modality Selection Strategies. In: Workshop on Spoken Language Technology (SLT). IEEE, Los Alamitos (2010)
Card, S.K., Mackinlay, J.D., Robertson, G.G.: The design space of input devices. In: Proc. SIGCHI 1990, pp. 117–124. ACM Press, New York (1990)
Chen, X., Tremaine, M.: Patterns of Multimodal Input Usage in Non-Visual Information Navigation. In: Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS 2006). IEEE, Los Alamitos (2006)
Bilici, V., Krahmer, E., Riele, S., Veldhuis, R.: Preferred Modalities in Dialogue Systems. In: Proc. ICSLP 2000, pp. 727–730 (2000)
Suhm, B., Myers, B., Waibel, A.: Model-based and empirical evaluation of multimodal interactive error correction. In: Proc. CHI 1999, pp. 584–591. ACM Press, New York (1999)
Naumann, A.B., Wechsung, I., Möller, S.: Factors Influencing Modality Choice in Multimodal Applications. In: Perception in Multimodal Dialogue, pp. 37–43 (2008)
Anderson, J.R., Bothell, D., Byrne, M.D., Douglass, S., Lebiere, C., Qin, Y.: An integrated theory of the mind. Psychological Review 111, 1036–1060 (2004)
Schaffer, S., Seebode, J., Wechsung, I., Metze, F., Möller, S.: Benutzerstudien zur Bewertung multimodaler, interaktiver Anzeigetafeln in unterschiedlichen Entwicklungsstufen. In: Kain, S., Struve, D., Wandke, H. (eds.) Workshop-Proceedings der Tagung Mensch und Computer 2009, pp. 22–27. Logos Berlin, Berlin (2009)
Perakakis, M., Potamianos, A.: Multimodal system evaluation using modality efficiency and synergy metrics. In: Proc. ICMI 2008, pp. 9–16. ACM Press, New York (2008)
Wechsung, I., Naumann, A.B., Möller, S.: Multimodale Anwendungen: Einflüsse auf die Wahl der Modalität. Mensch & Computer, 437–440 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Schaffer, S., Schleicher, R., Möller, S. (2011). A Model of Shortcut Usage in Multimodal Human-Computer Interaction. In: Duffy, V.G. (eds) Digital Human Modeling. ICDHM 2011. Lecture Notes in Computer Science, vol 6777. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21799-9_38
Download citation
DOI: https://doi.org/10.1007/978-3-642-21799-9_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21798-2
Online ISBN: 978-3-642-21799-9
eBook Packages: Computer ScienceComputer Science (R0)