Abstract
This paper describes an empirical study on what kinds of information are appropriate for referent identification requests in multi-modal dialogs, and how that information should be communicated in order to achieve the request desired. We conduct experiments in which experts explain the installation of a telephone in four situations: spoken-mode monolog; spoken-mode dialog; multi-modal monolog; and multi-modal dialog. Referent identification requests could be well analyzed from two perspectives: information communicated and the style of goal achievement. We find that there is a close relationship between the information conveyed via different communicative modes, and sketch a model that explains these results. In the model, information cannot be divided into the semantic content conveyed and the communicative modes employed, and is treated as the primitive unit for consideration. Pointing is considered as information in this sense. We also find that in dialogs, especially in spoken-mode dialogs, the speakers realize identification requests as series of fine-grained steps, and try to achieve them step by step.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
André, E. and Rist, T. (1994) Referring to World Objects with Text and Pictures, In Proceedings of COLING '94, 530–534.
Alshawi, H. (1987) Memory and Context for Language Interpretation, Cambridge: Cambridge University Press.
Appelt, D.E. (1985) Planning English Referring Expressions, Artificial Intelligence, 26, 1–33.
Clark, H.H. and Wilkes-Gibbs, D. (1990) Referring as a Collaborative Process. In Intentions in Communication, Cohen, P.R., Morgan, J. and Pollack, M.E. (eds.) The MIT Press, 463–493.
Claassen, W. (1992) Generating Referring Expressions in a Multimodal Environment.In Aspects of Automated Natural Language Generation, Hovy, R.D.O. Stock, D.R. (eds.) Heidelberg: Springer-Verlag, 247–262.
Cohen, P.R. (1984) The Pragmatics of Referring and the Modality of Communication, Computational Linguistics, 10(2), 97–146.
Feiner, S.K. and McKeown, K.R. (1990) Coordinating Text and Graphics in Explanation Generation. In Proceedings of AAAI-90, 442–449.
Grosz, B.J. and Sidner, C.L. (1986) Attention, Intentions, and the Structure of Discourse, Computational Linguistics, 12(3), 174–204.
Ishikawa, Y. (1984) Communicative Mode Dependent Contribution from the Recipient in Information Providing Dialogue. In Proceedings of ICSLP '94, 959–962.
Levelt, W.J.M. (1983) Monitoring and Self-Repair in Speech, Cognition, 14, 41–104.
Maybury, M.T. (1993) Planning Multimedia Explanations Using Communicative Acts. In Intelligent Multi Media Interfaces, The AAAI Press / The MIT Press, 60–74.
Neal, J.G. and Shapiro, S.C. (1991) Intelligent Multi-Media Technology. In Intelligent User Interfaces, Sullivan, J.W. and Tyler, S.W. (eds.) ACM Press, 11–43.
Oviatt, S.L. and Cohen, P.R. (1991) Discourse Structure and Performance Efficiency in Interactive and Noninteractive Spoken Modalities, Computer Speech and Language, 5(4), 297–326.
Wahlster, W., André, E., Graf, W. and Rist, T., Designing Illustrated Texts: How Language Production is Influenced by Graphics Generation. In Proceedings of EACL '91, 8–14.
Walker, M.A. (1992) Redundancy in Collaborative Dialogue. In Proceedings of 14th COLING, 345–351.
Walker, M.A. (1994) Experimentally Evaluating Communicative Strategies: The Effect of the Task. In Proceedings of AAAI-94, 86–93.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1998 Springer-Verlag
About this paper
Cite this paper
Kato, T., Nakano, Y.I. (1998). Referent identification requests in multi-modal dialogs. In: Bunt, H., Beun, RJ., Borghuis, T. (eds) Multimodal Human-Computer Communication. CMC 1995. Lecture Notes in Computer Science, vol 1374. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0052325
Download citation
DOI: https://doi.org/10.1007/BFb0052325
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64380-7
Online ISBN: 978-3-540-69764-0
eBook Packages: Springer Book Archive