Abstract
In this chapter we discuss our research on multimodal interaction in a virtual environment. The environment we have developed can be considered as a ‘laboratory’ for research on multimodal interactions and multimedia presentation, where we have multiple users and various agents that help the users to obtain and communicate information. The environment represents a theatre. The theatre has been built using VRML (Virtual Reality Modeling Language) and it can be accessed through World Wide Web (WWW). This virtual theatre allows navigation input through keyboard function keys and mouse, but there is also a navigation agent which tries to understand keyboard natural language input and spoken commands. Feedback of the system is given using speech synthesis. We also have Karen, an information agent which allows a natural language dialogue with the user. In development are several talking faces for the different agents in the virtual world. We investigate how we can increase the user’s commitment to the environment and its agents by providing context and increasing the user’s feeling of ‘presence’ in the environment.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Cassell, J., Pelachaud, C, Badler, N., Steedman, M., Achorn, B., Becket, T., Douville, B., Prevost, S. & M. Stone. Animated Conversation: Rule-Based Generation of Facial Expression, Gesture and Spoken Intonation for Multiple Conversational Agents. Proceedings of SIGGRAPH 94,1994.
Cassell, J. & K.R. Thorisson. The power of a nod and a glance: envelope vs. Emotional feedback in animated conversational agents. Applied Artificial Intelligence, to appear.
Cassell, J., O.E. Torres, S. Prevost. Turn taking vs. Discourse Structure: How Best to Model Multimodal Conversation. In: Wilks (ed.), Machine Conversations. The Hague: Kluwer, to appear.
Dirksen, A. and Menert, L. Fluent Dutch text-to-speech. Technical manual, Fluency Speech Technology/OTS Utrecht, 1997.
Doest, H. ter. Towards Probabilistic Unification-Based Parsing. Ph.D. Thesis, University Twente, February 1999.
Friedman, B. (ed.). Human Values and the Design of Computer Technology. CSLI Publications, Cambridge University Press, 1997.
Hulstijn, J. & A. van Hessen. Utterance Generation for Transaction Dialogues. Proceedings 5th International Conf. Spoken Language Processing (ICSLP), Vol. 4, Sydney, Australia, 1998,1143–1146.
Jonsson, A. Dialogue Management for Natural Language Interfaces. PhD thesis, Linkoping University, 1993.
Lie, D., J. Hulstijn, A. Nijholt, R. op den Akker. A Transformational Approach to NL Understanding in Dialogue Systems. Proceedings NLP and Industrial Applications, Moncton, New Brunswick, August 1998, 163–168.
Lombard, M. & T. Ditton. At the heart of it all: The concept of presence. Journal of Mediated Communication 3, Nr.2, September 1997.
Nass, C, B. Reeves & G. Leshner. Technology and roles: A tale of two TVs. Journal of Communication 46 (2), 121–128.
Sproull, L., M. Subramani, S. Kiesler, J. Walker & K. Waters. When the interface is a face. In [6], 163–190.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Nijholt, A., Hulstijn, J. (2000). Multimodal Interactions with Agents in Virtual Worlds. In: Kasabov, N. (eds) Future Directions for Intelligent Systems and Information Sciences. Studies in Fuzziness and Soft Computing, vol 45. Physica, Heidelberg. https://doi.org/10.1007/978-3-7908-1856-7_8
Download citation
DOI: https://doi.org/10.1007/978-3-7908-1856-7_8
Publisher Name: Physica, Heidelberg
Print ISBN: 978-3-7908-2470-4
Online ISBN: 978-3-7908-1856-7
eBook Packages: Springer Book Archive