Designing Multimodal Applications

  • David Cuka
  • Tasos Anastasakos
Part of the Computer Communications and Networks book series (CCN)


The vision of pervasive computing, the ability to access services and information anytime, anywhere, and from any device, is accelerated by emerging innovations in hardware, software, and network connectivity. Ubiquitous access presents an array of challenges, especially for applications in the mobile context because mobile devices have functional limitations due to size and available computational resources. Multimodal interfaces combined with careful considerations of the usability concerns in multimodal interaction can mitigate many of these limitations.


Mobile Device Markup Language Multimodal Interface Multimodal Interaction Voice Command 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    World Wide Web Consortium (W3C), Multimodal Interaction Activity
  2. 2.
    Open Mobile Alliance,
  3. 3.
    Cheyer, A., and Julia, L.(1995) Multimodal maps: An agent-based approach. In Proceedings of CMC95, Amsterdam, 1995, 103–113.Google Scholar
  4. 4.
    Johnston, M., Bangalore, S., and Vasireddy, G. (2001). MATCH: Multimodal Access to City Help. In Proceedings of ASRU 2001 Workshop, Madonna di Campiglio, Italy.Google Scholar
  5. 5.
    Almeida, L., Amdal, I., Beires, N., Boualem, M., Boves, L., den Os E., et al. (2002). Implementing and evaluating a multimodal and multilingual tourist guide. In Proceedings International CLASS workshop on natural, intelligent and effective interaction in Multimodal dialog system, Copenhagen, Denmark, June 2002.Google Scholar
  6. 6.
    Oviatt, S.L. (2000). Multimodal system processing in mobile environments. In Proceedings of the User Interface Software Technology Conference, Nov. 2000, San Diego, CA.Google Scholar
  7. 7.
    Oviatt, S.L. (2003). Multimodal interfaces. In The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, (ed. by J. Jacko and A. Sears), Lawrence Erlbaum Assoc., Mahwah, NJ, 2003, chap.14, 286–304.Google Scholar
  8. 8.
    Oviatt, S.L. (1999). Ten Myths of Multimodal Interaction, Communications of the ACM, 1999.Google Scholar
  9. 9.
    Seni, G., and Subrahmonia, J. (2001). Pen-Based user Interfaces — An Applications Overview. In The Computer Engineering Handbook, Oklobdzija V (ed), Series Electrical Engineering Handbook, vol.25, 2001.Google Scholar
  10. 10.
    Motorola, Lexicus Division. iTap.
  11. 11.
    Tegic Communications. T9.
  12. 12.
    Srinivasan, S., and Brown, E., (2002). Is Speech Recognition Becoming Mainstream? IEEE Computer Magazine, April 2002, 38–41.Google Scholar
  13. 13.
    Padmanabhan, M., and Picheny, M. (2002). Large-Vocabulary Speech Recognition Algorithms. IEEE Computer Magazine, April 2002, 42–50.Google Scholar
  14. 14.
    Gorin, A.L., Abella, X.X., Alonso, T., Riccardi, G., and Wright, J.H. (2002). Automated Natural Spoken Dialog, IEEE Computer Magazine, April 2002, 51–56.Google Scholar
  15. 15.
    World Wide Web Consortium (W3C), Voice Browser Activity,
  16. 16.
  17. 17.
    Ken Rehor’s World of VoiceXML,
  18. 18.
    Lum, W.Y., and Lau, F.C.M. (2002). A Context-Aware Decision Engine for Content Adaptation. In IEEE, Pervasive Computing, July-September, 2002, vol.1 No.3, p. 41–49.CrossRefGoogle Scholar
  19. 19.
    Oviatt, S., De Angeli, A., and Kuhn, K.(1997). Integration and synchronization of input modes during multimodal human-computer interaction. In Proceedings of the workshop Referring Phenomena in a Multimedia Context and their Computational Treatment, ACL/EACL’97, July 11, 1997, Madrid.Google Scholar
  20. 20.
  21. 21.
    Tang, S. Adding SALT to HTML,, May, 2003,
  22. 22.
    IBM Pervasive Computing Software, Multimodal Toolkit and documentation,
  23. 23.
    Maybury, M. (2001). Coordination and Fusion in Multimodal Interaction. MITRE, Technical Report, November 2001.Google Scholar
  24. 24.
    Norman, D.A. (1990). Why interfaces don’t work. In Laurel B (ed) The art of human-computer interface design, 1990, p. 209–219.Google Scholar
  25. 25.
    Weiss, D., and Hoffman, D. (eds) (2001). Software Fundamentals: Collected Papers by David L. Pamas.Google Scholar
  26. 26.
    US Department of Justice A Guide to Disability Rights Laws, May 2002,
  27. 27.
    Parnas, D.L. (1972). On the Criteria to Be Used in Decomposing Systems into Modules. Communications of the ACM, December 1972, vo1.15, No.12, p. 1053–105.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2004

Authors and Affiliations

  • David Cuka
  • Tasos Anastasakos

There are no affiliations available

Personalised recommendations