Advertisement

A software demonstrator of modality theory

  • N. O. Bernsen
  • S. Lu
Part of the Eurographics book series (EUROGRAPH)

Abstract

For some years, the multimodal systems group at the Centre for Cognitive Science, Roskilde University, has been working on establishing and implementing the research agenda of modality theory. The research agenda for modality theory is the following [1]:
  • 1. To establish sound conceptual and taxonomic foundations for describing and analysing any particular type of unimodal or multimodal output representation relevant to human-computer interaction (HCI);

  • 2. to create a conceptual framework for describing and analysing interactive computer interfaces;

  • 3. to develop a practical methodology for applying the results of steps (1) and (2) above to the problem of information mapping between work/task domains and human-computer interfaces in information systems design.

Keywords

Modality Theory Information Channel Taxonomy Tree Super Level Multimodal Representation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    N.O. Bernsen: A research agenda for modality theory. In Cox, R., Petre, M., Brna, P. and Lee, J. (Eds.): Proceedings of the Workshop on Graphical Representations, Reasoning and Communication. World Conference on Artificial Intelligence in Education, Edinburgh, 43–46, August 1993Google Scholar
  2. 2.
    N.O. Bernsen: Matching information and interface modalities. An example study. Esprit Basic Research project GRACE Deliverable 2.1. 1. (1993)Google Scholar
  3. 3.
    N.O. Bernsen: Modality theory in support of multimodal interface design. In Proceedings of the AAAI Spring Symposium on Intelligent Multi-Media Multi-Modal Systems, Stanford, March 1994, 37–44Google Scholar
  4. 4.
    N.O. Bernsen: Foundations of multimodal representations. A taxonomy of representational modalities. Interacting with Computers Vol. 6, 4, 347–71. (1994)CrossRefGoogle Scholar
  5. 5.
    N.O. Bernsen: A revised generation of the taxonomy of output modalities. Esprit Basic Research project AMODEUS-2 Working Paper RP5–TMWP11, (1994)Google Scholar
  6. 6.
    N.O. Bernsen: Why are analogue graphics and natural language both needed in HCI? In Paterno, F. (Ed.): Design, Specification and Verification of Interactive Systems. Proceedings of the Eurographics Workshop, Carrara, Italy, June, 165–179. Forthcoming in the Focus on Computer Graphics Series, Springer Verlag 1995Google Scholar
  7. 7.
    N.O. Bernsen and A. Bertels: A methodology for mapping information from task domains to interactive modalities. Esprit Basic Research project GRACE Deliverable 10.1. 3. (1993)Google Scholar
  8. 8.
    N.O. Bernsen, S. Lu and M. May: Towards a design support tool for multimodal interface design. The taxonomy workbench and theory demonstrator. Esprit Basic Research project AMODEUS-2 Working Paper RP5-TM-WP5. (1994)Google Scholar
  9. 9.
    N.O. Bernsen and S. Verjans: Designing interfaces by information mapping. A case study of the CERD design. Esprit Basic Research project AMODEUS-2 Working Paper (forthcoming). (1995)Google Scholar
  10. 10.
    E. Hovy and Y. Arens: When is a picture worth a thousand words ? Allocation of modalities in multimedia communication. Paper presented at the AAAI Symposium on Human-Computer Interfaces, Stanford (1990)Google Scholar
  11. 11.
    A. Lockwood: Diagrams. A visual survey of graphs, maps, charts and diagrams for the graphic designer. London: Studio Vista (1969)Google Scholar
  12. 12.
    M. May: Representations and homomorphisms. A taxonomy of representations for HCI, part 1. Esprit Basic Research project GRACE Deliverable 2.1. 5 (1993)Google Scholar
  13. 13.
    M. May: Levels of representations and mappings. A taxonomy of representations for HCI, part 2. Esprit Basic Research project GRACE Deliverable 2.1. 6. (1993)Google Scholar
  14. 14.
    M. May: From semantic types to multimodal presentations: Case studies. A taxonomy of representations for HCI, part 3. Esprit Basic Research project GRACE Deliverable 2.1. 7. (1993)Google Scholar
  15. 15.
    M. May: Taxonomy and levels of mappings in the construction of multimodal interfaces. Workshop Notes from AI-ED ‘83: World Conference on Artificial Intelligence in Education, Edinburgh, August. Workshop on Graphical Representations, Reasoning and Communication. (1993)Google Scholar
  16. 16.
    M. May and N.O. Bernsen: The taxonomy workbench. Esprit Basic Research project GRACE Deliverable 2.1. 8. (1993)Google Scholar
  17. 17.
    M. May and R. Tobin: The taxonomy workbench. Purpose, functionality and layout. Esprit Basic Research project GRACE Deliverable 2.1. 9. (1993)Google Scholar
  18. 18.
    E.R. Tufte: Envisioning information. Cheshire: Graphics Press 1990Google Scholar
  19. 19.
    S. Verjans: EuroCODE: Preliminary information mapping analysis of an AMODEUS common exemplar. Esprit Basic Research project AMODEUS-2 Internal Report RP5-TM-IR1. (1994)Google Scholar
  20. 20.
    S. Verjans and N.O. Bernsen: PaTerm: A case study in information mapping. Esprit Basic Research project AMODEUS-2 Working Paper RP5TM-WP6. (1994)Google Scholar

Copyright information

© Springer-Verlag/Wien 1995

Authors and Affiliations

  • N. O. Bernsen
    • 1
  • S. Lu
    • 1
  1. 1.Centre for Cognitive ScienceRoskilde UniversityRoskildeDenmark

Personalised recommendations