Interaction Models, Reference, and Interactivity in Speech Interfaces to Virtual Environments

  • Jussi Karlgren
  • Ivan Bretan
  • Niklas Frost
  • Lars Jonsson
Part of the Eurographics book series (EUROGRAPH)


The enhancement of a virtual reality environment with a speech interface is described. Some areas where the virtual reality environment benefits from the spoken modality are identified as well as some where the interpretation of natural language utterances benefits from being situated in a highly structured environment. The issue of interaction metaphors for this configuration of interface modalities is investigated.


Virtual Environment Multimodal Interface Visual Awareness Discourse Referent Collaborative Virtual Environment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Benford, Steve, John Bowers, Lennart Fahlén, and Chris Greenhalgh. 1994. “Managing Mutual Awareness in Collaborative Virtual Environments” Proceedings of VRST’94, Singapore. New York: ACM.Google Scholar
  2. 2.
    Benford, Steve, John Bowers, Lennart Fahlén, and Chris Greenhalgh. 1995. “User Embodiment in Collaborative Virtual Environments” Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI’95), Boston. New York: ACM.Google Scholar
  3. 3.
    Benford, Steve and Lennart Fahlén. 1993. “A Spatial Model of Interaction in Large Virtual Environments” Proceedings of 3d ECSCWMilm: Kluwer.Google Scholar
  4. 4.
    Bos, Edwin, Carla Hüls, and Wim Ciaassen. 1994. “EDWARD: full integration of language and action in a multimodal user interface” International Journal of Human-Computer Studies, 40: 473–495.CrossRefGoogle Scholar
  5. 5.
    Bretan, Ivan and Jussi Karlgren. 1993. “Synergy Effects In Natural Language-Based Multimodal Interfaces” Proceedings of 1993 ERCIM Workshop on Multimodal Human-Computer Interaction, Nancy: INRIA. (also available as SICS Research Report R94: 04.Google Scholar
  6. 6.
    Bretan, Ivan and Jussi Karlgren. 1994. “Worlds without Words”, Proceedings of ERCIM Workshop on VR, Stockholm: SICS.Google Scholar
  7. 7.
    Brown, Gillian and George Yule. 1983. Discourse Analysis. Cambridge: Cam-bridge University Press.Google Scholar
  8. 8.
    Carlsson, Christer and Olof Hagsand. 1993. “DIVE, a Platform for Multi-User Virtual Environments”, Computers & Graphics, 17: 6.CrossRefGoogle Scholar
  9. 9.
    Chandrasekar, R. and S. Ramani. 1989. “Interactive communication of sentential structure and content: an alternative approach to man-machine communication”, International Journal of Man-Machine Studies 30: 121–148.CrossRefGoogle Scholar
  10. 10.
    Cohen, Philip. 1992. “The Role of Natural Language in a Multimodal Interface”, In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST), Monterey: ACM.Google Scholar
  11. 11.
    Dahlbäck, Nils, Arne Jönsson, and Lars Ahrenberg. 1993. “Wizard-of-Oz Studies-Why and How”, Proceedings of the 1993 International Workshop on Intelligent User Interfaces, Orlando: ACM.Google Scholar
  12. 12.
    Fahlén, Lennart, Charles G. Brown, Olov Stähl, Christer Carlsson. 1993. “A Space Based Model for User Interaction in Shared Synthetic Environments” Proceedings of the ACM Conference on Human Factors in Computing Systems (InterCHI’93) Amsterdam:ACM.Google Scholar
  13. 13.
    Fraurud, Kari. 1990. “Definiteness and the Processing of NP’s in Natural Discourse” Journal of Semantics 7: 395–433.CrossRefGoogle Scholar
  14. 14.
    Joshi, Aravind. 1982. “Mutual Beliefs in Question-Answering Systems”, in N. V. Smith (ed), Mutual Knowledge, London: Academic Press.Google Scholar
  15. 15.
    Karlgren, Jussi, Kristina Höök, Ann Lantz, Jacob Palme, and Daniel Pargman. 1994. “The Glass Box User Model for Information Filtering”, Proceedings of the 4th International Conference on User Modeling Cape Cod: ACM. ( A longer version available as SICS Technical Report T94. 09 ).Google Scholar
  16. 16.
    Karlsson, Fred. 1990. “Constraint Grammar for Parsing Running Text”. Papers presented to the Thirteenth International Conference On Computational Linguistics (COLING-90), H. Karlgren (ed.), Helsinki: University of Helsinki.Google Scholar
  17. 17.
    Karlsson, Fred, Atro Voutilainen, Juha Heikkilä, and Arto Anttila (eds.) 1995. Constraint Grammar Berlin: Mouton de Gruyter.Google Scholar
  18. 18.
    Lauri Karttunen. 1969 (1976). “Discourse Referents”. Paper presented to the International Conference On Computational Linguistics (COLING-69), Sänga-Säby. Stockholm:KVAL. Also in James D. McCawley (ed.) Notes from the Linguistic Underground. Syntax and Semantics, Vol 7. Pp. 363–385. New York: Academic Press.Google Scholar
  19. 19.
    Lemaire, Benoit and Johanna Moore. 1994. “An Improved Interface for Tutorial Dialogues: Browsing a Visual Dialogue History”. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI’94), Boston: ACM.Google Scholar
  20. 20.
    LuperFoy, Susann. 1991. Discourse PEGS: A Computational Analysis of Context-Dependent Referring Expressions. Ph D Dissertation. Austin: University of Texas at Austin.Google Scholar
  21. 21.
    Woodland, P. C., J. J. Odell, V. Valtchev and S. J. Young. 1994. “Large Vocabulary Continuous Speech Recognition Using HTK”. Proceedings of ICASSP’94, Adelaide.Google Scholar
  22. 22.
    Zoltan-Ford, Elizabeth. 1991. “How to get people to say and type what computers can understand”. International Journal of Man-Machine Studies 34: 527–547.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag/Wien 1995

Authors and Affiliations

  • Jussi Karlgren
    • 1
  • Ivan Bretan
    • 1
  • Niklas Frost
    • 1
  • Lars Jonsson
    • 1
  1. 1.Swedish Institute of Computer ScienceKista, StockholmSweden

Personalised recommendations