Human-computer graphical dialogue
One of the goals of Human-Computer Interaction (HCI) researchers is to design and implement intelligent interfaces which will allow users to converse with the underlying computer system in a natural manner. "Human beings communicate using a wide variety of forms including natural language, both spoken and written, pictures, images, documents, and the like. It is difficult for current computers to understand these natural input/output forms and to respond to them intelligently, since they are not equipped with intelligent man-machine interfaces" (Tanaka, Chiba, et al 1982).
In the past twenty years Artificial Intelligence (AI) researchers have put much effort into solving the problems that exist in Natural Language (NL) interfaces. Unfortunately many of these problems are still unresolved, and to-date semantic grammars as used by systems such as, SOPHIE, LIFER and PLANES (Hendrix et al, 1978) remain the most efficient way of implementing a NL interface.
This paper discusses an experiment which indicates that graphical interfaces can offer users a more effective means of communicating their intentions to the system than is possible with NL, as instead of typing long sentences, the user simply points to and clicks on objects that appear on the screen. An object oriented prototype, Circuit (Singer, 1989), has shown that certain discourse phenomena (anaphora and ellipsis) which are handled by SOPHIE (Brown, Burton et al, 1974, 1976, 1982) can be modelled in this way. It demonstrates the feasibility of an interface which allows users to express their thoughts in terms of manipulating graphical objects rather than NL.
The relationships between user thoughts and graphical objects, if they are to be natural and effective, reflect the structure of human discourse. This requires an interface which can understand the relation of subsequent thoughts to preceding ones (Reichman, 1984). To do this the interface should construct a discourse model of the conversation as it proceeds. The construction of ‘context spaces’ within such models are necessary for the resolving of anaphora and ellipsis which contribute to the smoothness and coherency of a dialogue.
The success of this experiment has clearly demonstrated that the notion of embedding such discourse capabilities within a graphics environment is a viable alternative to that of NL given the current unresolved problems.
KeywordsGraphical Interface Intelligent Tutor System Graphical Treatment Graphical Object Focus Level
Unable to display preview. Download preview PDF.
- Bonar, J.G., and Weil, W. (1985) An informal programming language. Paper presented at the meeting Expert Systems in Government, October, Washington, D.C.Google Scholar
- Brown, J.S., Burton, R.R., and Bell, A.G. (1974) SOPHIE A Sophisticated Instructional Environment for Teaching Electronic Troubleshooting. BBN Report No. 2790. Bolt Beranek and Newman Inc., Cambridge, Massachusetts.Google Scholar
- Brown, J.S., Burton, R.R., and Rubenstein, R. (1976) Reactive Learning Environment for Computer Assisted Electronics Instruction. BBN Report 3314. Bolt Beranek and Newman Inc., Cambridge, Massachusetts.Google Scholar
- Brown, J.S., Burton, R.R., deKleer, J. (1982) Pedagogical, natural language and knowledge engineering techniques in SOPHIE I, II and III. In Sleeman, D. and Brown, J.S. (Eds.) Intelligent Tutoring Systems, p. 243. Academic Press., 1982.Google Scholar
- Burton, R.R., and Brown, J.S. (1979) Towards a Natural-Language Capability for Computer-Assisted Instruction. In Natural Language Processing, p. 612. Morgan Kaufmann Publishers., 1986Google Scholar
- Feigenbaum, E.A. (1974) Artificial Intelligence Research: What is it? What has it achieved? Where is it going? Symposium on Artificial Intelligence, Canberra, Australia.Google Scholar
- Grosz, B.J., Jones, K.S., and Webber, B.L. (Eds.) (1986) Readings in Natural Language Processing. Morgan Kaufmann Publishers.Google Scholar
- Hendrix, G.G., Earl, D., Sacerdoti., Sagalowicz, D., and Slocum, J. (1978) Developing a Natural Language Interface to complex data. In Readings in Natural Language Processing edited by Grosz, B.J., Jones, K.S., and Webber, B.L. (1986). Morgan Kaufmann Publishers.Google Scholar
- Hollan, J.D., Hutchins, E.L., Weitzman, L.M. (1984) STEAMER: An Interactive, Inspectable, Simulation-Based Training System. The AI Magazine (Summer, 1984), Published by the American Association for Artificial Intelligence.Google Scholar
- Meadow, C.T. (1970) Man-Machine Communication Center for Computer Sciences and Technology. National Bureau of Standards.Google Scholar
- Petrick, S.R., (1976). On Natural Language Based Computer Systems. In IBM Journal Research & Development, 20, 314–325. Excerpt taken from Human-Computer Communication, Smith, H.T.(1980) p. 29–30. In Human Interaction with Computers, Smith and Green (Eds.) (1980). Academic Press Inc. London Ltd.Google Scholar
- Psotka, J., Massey, D.L., Mutter, S.A. (Eds.) (1988) Intelligent Tutoring Systems, Lessons Learned, p 404. Lawrence Erlbaum Associates Publishers, Hillsdale, New Jersey.Google Scholar
- Reichman, R. (1986) Communication Paradigms for a Window System. In Norman, D., and Draper, S.W. (Eds.) User-Centered System Design: New Perspectives on Human-Computer Interaction. Lawrence Erlbaum Associates, Hillsdale, New Jersey.Google Scholar
- Sidner, C.L. (1986) Focusing in the Comprehension of Definite Anaphora. In Natural Language Processing, p. 363.. Morgan Kaufmann Publishers.Google Scholar
- Simply Stated. (1987) Using Icons as Communication. No. 75, September/October issue. The newsletter of the Document Design Center, American Institutes for Research.Google Scholar
- Singer, R.A. (1989) Graphical Treatment of Anaphora & Ellipsis in HCI. CITE Report No. 94. Open University publication.Google Scholar
- Smith, H.T. (1980) Human-Computer Communication. In Smith, H.T., Green, T.R.G. (Eds.) Human Interaction with Computers. Academic Press.Google Scholar
- Tanaka, H., Chiba, S., Kidide, M., Tamura, H., Kodera, T. (1982) Intelligent Man-Machine Interface, p. 147. Fifth Generation Computer Systems, edited by T. Moto-oka. North-Holland Publishing Company. JIPDEC, 1982.Google Scholar
- Towne, D.M., Munro, A., Pizzini, Q.A., and Surmon, D.S. (1985) Development of intelligent maintenance training technology: Design study. Technical report no. 106). Los Angeles: Behavioural Technology Laboratories, University of Southern California.Google Scholar
- White, B.Y., and Frederiksen, J.R. (1984) Modelling expertise in troubleshooting and reasoning about simple electric circuits. Proceedings of the Sixth Cognitive Science Society Conference, Boulder, Colorado., pp. 337–343.Google Scholar
- White, B.Y., and Frederiksen, J.R. (1985) QUEST: qualitative models and intelligent learning environments. In Lawler, R., and Yazadani, M. (Eds.) AI and Education: Learning Environments and Intelligent Tutoring Systems. Ablex Publishing, Norwood, New Jersey.Google Scholar