Multimodal interfaces are characterized by the (possibly simultaneous) use of multiple human sensory modalities and can support combined input/output modes.
The term multimodal recurs across several domains. Since its first use in the field of interface design, its affinity and derivation from the terms “mode” and “modality” were discussed. According to Merriam-Webster, one of the meanings for mode is “a possible, customary, or preferred way of doing something,” whereas modality can be “one of the main avenues of sensation (as vision).” In multimodal interfaces, the former influences the way information is conveyed, the latter refers to the exploited communication channel. Both express peculiar aspects of a multimodal system, which is expected to provide users with flexibility and natural interaction.
Early multimodal interfaces simply combined display, keyboard, and mouse with voice (speech recognition/synthesis). Later, pen-based input, hand gestures, eye gaze, haptic...
- 3.Cohen PR, Johnston M, McGee DR, Oviatt SL, Pittman J, Smith I, Chen L, Clow J. QuickSet: multimodal interaction for distributed applications. In: Proceedings of the 5th ACM International Conference on Multimedia; 1997. p. 31–40.Google Scholar
- 4.European Telecommunications Standards Institute. Human Factors (HF); Multimodal interaction, communication and navigation guidelines ETSI EG 202 191 V1.1.1 (2003–08).Google Scholar
- 6.Norman D. The design of everyday things. New York: Doubleday; 1988.Google Scholar
- 8.Oviatt S. Multimodal interfaces. In: Jacko J, Sears A, editors. The human-computer interaction handbook: fundamentals, evolving technologies, and emerging applications. Mahwah: Lawrence Erlbaum; 2003.Google Scholar