Human-computer graphical dialogue

  • Ronald A. Singer
AI Applications In CAL
Part of the Lecture Notes in Computer Science book series (LNCS, volume 438)


One of the goals of Human-Computer Interaction (HCI) researchers is to design and implement intelligent interfaces which will allow users to converse with the underlying computer system in a natural manner. "Human beings communicate using a wide variety of forms including natural language, both spoken and written, pictures, images, documents, and the like. It is difficult for current computers to understand these natural input/output forms and to respond to them intelligently, since they are not equipped with intelligent man-machine interfaces" (Tanaka, Chiba, et al 1982).

In the past twenty years Artificial Intelligence (AI) researchers have put much effort into solving the problems that exist in Natural Language (NL) interfaces. Unfortunately many of these problems are still unresolved, and to-date semantic grammars as used by systems such as, SOPHIE, LIFER and PLANES (Hendrix et al, 1978) remain the most efficient way of implementing a NL interface.

This paper discusses an experiment which indicates that graphical interfaces can offer users a more effective means of communicating their intentions to the system than is possible with NL, as instead of typing long sentences, the user simply points to and clicks on objects that appear on the screen. An object oriented prototype, Circuit (Singer, 1989), has shown that certain discourse phenomena (anaphora and ellipsis) which are handled by SOPHIE (Brown, Burton et al, 1974, 1976, 1982) can be modelled in this way. It demonstrates the feasibility of an interface which allows users to express their thoughts in terms of manipulating graphical objects rather than NL.

The relationships between user thoughts and graphical objects, if they are to be natural and effective, reflect the structure of human discourse. This requires an interface which can understand the relation of subsequent thoughts to preceding ones (Reichman, 1984). To do this the interface should construct a discourse model of the conversation as it proceeds. The construction of ‘context spaces’ within such models are necessary for the resolving of anaphora and ellipsis which contribute to the smoothness and coherency of a dialogue.

The success of this experiment has clearly demonstrated that the notion of embedding such discourse capabilities within a graphics environment is a viable alternative to that of NL given the current unresolved problems.


Graphical Interface Intelligent Tutor System Graphical Treatment Graphical Object Focus Level 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bonar, J.G., and Weil, W. (1985) An informal programming language. Paper presented at the meeting Expert Systems in Government, October, Washington, D.C.Google Scholar
  2. Brown, J.S., Burton, R.R., and Bell, A.G. (1974) SOPHIE A Sophisticated Instructional Environment for Teaching Electronic Troubleshooting. BBN Report No. 2790. Bolt Beranek and Newman Inc., Cambridge, Massachusetts.Google Scholar
  3. Brown, J.S., Burton, R.R., and Rubenstein, R. (1976) Reactive Learning Environment for Computer Assisted Electronics Instruction. BBN Report 3314. Bolt Beranek and Newman Inc., Cambridge, Massachusetts.Google Scholar
  4. Brown, J.S., Burton, R.R., deKleer, J. (1982) Pedagogical, natural language and knowledge engineering techniques in SOPHIE I, II and III. In Sleeman, D. and Brown, J.S. (Eds.) Intelligent Tutoring Systems, p. 243. Academic Press., 1982.Google Scholar
  5. Burton, R.R., and Brown, J.S. (1979) Towards a Natural-Language Capability for Computer-Assisted Instruction. In Natural Language Processing, p. 612. Morgan Kaufmann Publishers., 1986Google Scholar
  6. Feigenbaum, E.A. (1974) Artificial Intelligence Research: What is it? What has it achieved? Where is it going? Symposium on Artificial Intelligence, Canberra, Australia.Google Scholar
  7. Grosz, B.J., Jones, K.S., and Webber, B.L. (Eds.) (1986) Readings in Natural Language Processing. Morgan Kaufmann Publishers.Google Scholar
  8. Hendrix, G.G., Earl, D., Sacerdoti., Sagalowicz, D., and Slocum, J. (1978) Developing a Natural Language Interface to complex data. In Readings in Natural Language Processing edited by Grosz, B.J., Jones, K.S., and Webber, B.L. (1986). Morgan Kaufmann Publishers.Google Scholar
  9. Hollan, J.D., Hutchins, E.L., Weitzman, L.M. (1984) STEAMER: An Interactive, Inspectable, Simulation-Based Training System. The AI Magazine (Summer, 1984), Published by the American Association for Artificial Intelligence.Google Scholar
  10. Meadow, C.T. (1970) Man-Machine Communication Center for Computer Sciences and Technology. National Bureau of Standards.Google Scholar
  11. Petrick, S.R., (1976). On Natural Language Based Computer Systems. In IBM Journal Research & Development, 20, 314–325. Excerpt taken from Human-Computer Communication, Smith, H.T.(1980) p. 29–30. In Human Interaction with Computers, Smith and Green (Eds.) (1980). Academic Press Inc. London Ltd.Google Scholar
  12. Psotka, J., Massey, D.L., Mutter, S.A. (Eds.) (1988) Intelligent Tutoring Systems, Lessons Learned, p 404. Lawrence Erlbaum Associates Publishers, Hillsdale, New Jersey.Google Scholar
  13. Reichman, R. (1978) Conversational Coherency. Cognitive Science 2, 283–327.CrossRefGoogle Scholar
  14. Reichman, R. (1984) Extended Person-Machine Interface. Artificial Intelligence 22, p. 157 Elsevier Science Publishers B.V. (North Holland).CrossRefGoogle Scholar
  15. Reichman, R. (1986) Communication Paradigms for a Window System. In Norman, D., and Draper, S.W. (Eds.) User-Centered System Design: New Perspectives on Human-Computer Interaction. Lawrence Erlbaum Associates, Hillsdale, New Jersey.Google Scholar
  16. Sidner, C.L. (1986) Focusing in the Comprehension of Definite Anaphora. In Natural Language Processing, p. 363.. Morgan Kaufmann Publishers.Google Scholar
  17. Simply Stated. (1987) Using Icons as Communication. No. 75, September/October issue. The newsletter of the Document Design Center, American Institutes for Research.Google Scholar
  18. Singer, R.A. (1989) Graphical Treatment of Anaphora & Ellipsis in HCI. CITE Report No. 94. Open University publication.Google Scholar
  19. Smith, H.T. (1980) Human-Computer Communication. In Smith, H.T., Green, T.R.G. (Eds.) Human Interaction with Computers. Academic Press.Google Scholar
  20. Tanaka, H., Chiba, S., Kidide, M., Tamura, H., Kodera, T. (1982) Intelligent Man-Machine Interface, p. 147. Fifth Generation Computer Systems, edited by T. Moto-oka. North-Holland Publishing Company. JIPDEC, 1982.Google Scholar
  21. Towne, D.M., Munro, A., Pizzini, Q.A., and Surmon, D.S. (1985) Development of intelligent maintenance training technology: Design study. Technical report no. 106). Los Angeles: Behavioural Technology Laboratories, University of Southern California.Google Scholar
  22. White, B.Y., and Frederiksen, J.R. (1984) Modelling expertise in troubleshooting and reasoning about simple electric circuits. Proceedings of the Sixth Cognitive Science Society Conference, Boulder, Colorado., pp. 337–343.Google Scholar
  23. White, B.Y., and Frederiksen, J.R. (1985) QUEST: qualitative models and intelligent learning environments. In Lawler, R., and Yazadani, M. (Eds.) AI and Education: Learning Environments and Intelligent Tutoring Systems. Ablex Publishing, Norwood, New Jersey.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1990

Authors and Affiliations

  • Ronald A. Singer
    • 1
  1. 1.Institute of Educational Technology, Centre for Information Technology in EducationThe Open UniversityMilton KeynesU.K.

Personalised recommendations