A Distributed Staged Architecture for Multimodal Applications

  • Alessandro Costa Pereira
  • Falk Hartmann
  • Kay Kadner
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4758)


Most of the research in the area of multimodality discusses either the usability aspect of multimodality or the multimodality support given by or missing in certain markup languages. The overall architectural side of large multimodal systems is unfortunately not adequately represented in today’s literature. This report shows some results obtained during the implementation of such a system, e.g., the use of the multimodal interaction framework and how a staged architecture can be combined with this framework to achieve domain independence.


Gesture Recognition Markup Language Architectural Style Multimodal Interface Multimodal Interaction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Pereira, A.C., Hartmann, F., Kadner, K.: A Distributed Staged Architecture for Multimodal Applications (extended Abstract). In: SE 2007. Software Engineering 2007. Lecture Notes in Informatics (LNI), vol. 105, Köllen Verlag, Bonn (2007)Google Scholar
  2. 2.
    The SNOW Consortium: SNOW Project Homepage (2005) (visited May 10, 2006),
  3. 3.
    The World Wide Web Consortium: EMMA: Extensible MultiModal Annotation markup language (2005) (visited June 2, 2006),
  4. 4.
    The Internet Engineering Task Force: A Media Resource Control Protocol (MRCP) (2006) (visited October 4, 2006),
  5. 5.
    The World Wide Web Consortium: Speech Synthesis Markup Language (SSML) Version 1.0 (2004) (visited October 4, 2006),
  6. 6.
    The Fundamental Modeling Concepts Consortium: Fundamental Modeling Concepts (2003) (visited May 29, 2006),
  7. 7.
    Knöpfel, A.: FMC quick introduction. FMC Publication (2003),
  8. 8.
    Coutaz, J., Nigay, L., Salber, D., Blandford, A., May, J., Young, R.M.: Four Easy Pieces for Assessing the Usability of Multimodal Interaction: The CARE Properties. In: Proceedings of INTERACT 1995, pp. 115–120 (1995)Google Scholar
  9. 9.
    Nigay, L., Coutaz, J.: Multifeature Systems: The CARE Properties and Their Impact on Software Design. AAAI Press, Stanford, California, USA (1997)Google Scholar
  10. 10.
    The World Wide Web Consortium: Multimodal Interaction Framework (2003) (visited M ay 29, 2006),
  11. 11.
    Wahlster, W. (ed.): SmartKom: Foundations of Multimodal Dialogue Systems. Cognitive Technologies. Springer, Heidelberg (2006)Google Scholar
  12. 12.
    The World Wide Web Consortium: Multimodal Interaction Requirements (2003) (visited May 31, 2006),
  13. 13.
    Bolt, R.A.: Voice and gesture at the graphics interface. In: SIGGRAPH 1980. Proceedings of the 7th annual conference on Computer graphics and interactive techniques, pp. 262–270. ACM Press, New York (1980)CrossRefGoogle Scholar
  14. 14.
    Riehle, D., Gross, T.: Role model based framework design and integration, pp. 117–133. ACM Press, New York (1998)Google Scholar
  15. 15.
    Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, Reading, Massachusetts (1995)Google Scholar
  16. 16.
    Kadner, K.: A flexible architecture for multimodal applications using federated devices. Visual Languages and Human-Centric Computing, 236–237 (2006)Google Scholar
  17. 17.
    Aßmann, U.: Architectural styles for active documents. Science of Computer Programming 56, 79–98 (2005)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Ossher, H., Tarr, P.: Multi-dimensional separation of concerns and the hyperspace approach. In: Proceedings of the Symposium on Software Architectures and Component Technology: The State of the Art in Software Development, Kluwer, Dordrecht (2000)Google Scholar
  19. 19.
    Kadner, K., Roussel, D.: Documentation for aircraft maintenance based on topic maps. In: Maicher, L., Sigel, A., Garshol, L.M. (eds.) TMRA 2006. LNCS (LNAI), vol. 4438, pp. 56–61. Springer, Heidelberg (2007)Google Scholar
  20. 20.
    Göbel, S., Hartmann, F., Kadner, K., Pohl, C.: A device-independent multimodal mark-up language. In: INFORMATIK 2006: Informatik für Menschen, Band 2, pp. 170–177 (2006)Google Scholar
  21. 21.
    World Wide Web Consortium: XHTMLTM1.0 The Extensible HyperText Markup Language, 2nd edn. (2000) (visited May14, 2006),
  22. 22.
    Carlsson, V., Klug, T., Ziegert, T., Zinnen, A.: Wearable computers in clinical ward rounds. In: IFAWC 2006. Proceedings of the third International Forum on Applied Wearable Computing, pp. 45–53 (2006)Google Scholar
  23. 23.
    Aßmann, U.: Invasive Software Composition. Springer, Heidelberg (2003)zbMATHGoogle Scholar
  24. 24.
    Hartmann, F.: An architecture for an xml-template engine enabling safe authoring. In: DEXA 2006, pp. 502–507. IEEE Computer Society, Washington, DC, USA (2006)Google Scholar
  25. 25.
    Cohen, P.R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., Clow, J.: Quickset: Multimodal interaction for distributed applications. In: Proceedings of ACM Multimedia 1997, pp. 31–40. ACM Press, New York (1997)CrossRefGoogle Scholar
  26. 26.
    Elting, C., Rapp, S., Möhler, G., Strube, M.: Architecture and implementation of multimodal plug and play. In: ICMI 2003. Proceedings of the 5th international conference on Multimodal interfaces, pp. 93–100. ACM Press, New York (2003)CrossRefGoogle Scholar
  27. 27.
    West, D., Apted, T., Quigley, A.: A context inference and multi-modal approach to mobile information access. In: Artificial Intelligence in Mobile Systems, Nottingham, England, pp. 28–35 (2004)Google Scholar
  28. 28.
    Bouchet, J., Nigay, L., Ganille, T.: Icare software components for rapidly developing multimodal interfaces. In: ICMI 2004. Proceedings of the 6th international conference on Multimodal interfaces, pp. 251–258. ACM Press, New York (2004)CrossRefGoogle Scholar
  29. 29.
    Kost, S.: Dynamically generated multi-modal application interfaces. PhD thesis, TU Dresden (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Alessandro Costa Pereira
    • 1
    • 2
  • Falk Hartmann
    • 1
    • 2
  • Kay Kadner
    • 1
    • 2
  1. 1.Department of Computer Science, Technische Universität Dresden, DresdenGermany
  2. 2.SAP Research CEC Dresden, SAP AG, Dresden, Germany, 

Personalised recommendations