Human-computer graphical dialogue

Abstract

One of the goals of Human-Computer Interaction (HCI) researchers is to design and implement intelligent interfaces which will allow users to converse with the underlying computer system in a natural manner. "Human beings communicate using a wide variety of forms including natural language, both spoken and written, pictures, images, documents, and the like. It is difficult for current computers to understand these natural input/output forms and to respond to them intelligently, since they are not equipped with intelligent man-machine interfaces" (Tanaka, Chiba, et al 1982).

In the past twenty years Artificial Intelligence (AI) researchers have put much effort into solving the problems that exist in Natural Language (NL) interfaces. Unfortunately many of these problems are still unresolved, and to-date semantic grammars as used by systems such as, SOPHIE, LIFER and PLANES (Hendrix et al, 1978) remain the most efficient way of implementing a NL interface.

This paper discusses an experiment which indicates that graphical interfaces can offer users a more effective means of communicating their intentions to the system than is possible with NL, as instead of typing long sentences, the user simply points to and clicks on objects that appear on the screen. An object oriented prototype, Circuit (Singer, 1989), has shown that certain discourse phenomena (anaphora and ellipsis) which are handled by SOPHIE (Brown, Burton et al, 1974, 1976, 1982) can be modelled in this way. It demonstrates the feasibility of an interface which allows users to express their thoughts in terms of manipulating graphical objects rather than NL.

The relationships between user thoughts and graphical objects, if they are to be natural and effective, reflect the structure of human discourse. This requires an interface which can understand the relation of subsequent thoughts to preceding ones (Reichman, 1984). To do this the interface should construct a discourse model of the conversation as it proceeds. The construction of ‘context spaces’ within such models are necessary for the resolving of anaphora and ellipsis which contribute to the smoothness and coherency of a dialogue.

The success of this experiment has clearly demonstrated that the notion of embedding such discourse capabilities within a graphics environment is a viable alternative to that of NL given the current unresolved problems.