The Problems of Integrating Interfaces to Publicly Available Databases

  • Jane Littlehales
  • Peter Hancox
Conference paper
Part of the Workshops in Computing book series (WORKSHOPS COMP.)


This paper looks at the drawbacks of existing user interface mechanisms for accessing databases, namely query languages, natural language and Query by Example, in relation to publicly available database systems. An integrated, multimodal interface, where natural language and pointing gestures to a graphical display can be combined into a single sentence, using deictic words such as this, here, or used individually, is proposed. This is seen as closely imitating human dialogue. The application chosen is a geographical database, taking advantage of a familiar atlas metaphor. The progress to date is described with results from the first stage of integration. A number of existing combined mode systems are analysed, particularly with respect to their degree of integration, gestund ambiguity and other error handling facilities.


Natural Language Query Language Graphical Interaction Graphical Output Multimodal Interface 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [Allg86]
    Allgayer J, Reddig C. Processing Descriptions Containing Words and Gestures — A System Architecture, In: Proceedings of GWAI —’ 86 und Österreichische Artificial-Intelligence-Tagung, Ottenstein, Austria, September, 1986, Springer-Verlag, pp 119-130Google Scholar
  2. [Allg89]
    Allgayer J, Harbusch K, Kobsa A, Reddig C, Reithinger N, Schmauks D. XTRA — A Natural Language Access System to Expert Systems, International Journal of Man-machine Studies 1989; 31: 611–195CrossRefGoogle Scholar
  3. [Bino88]
    Binot J-L, Demoen B, Hanne K-H, et al. LOKI: A Logic Oriented Approach to Data and Knowledge Bases supporting Natural Language Interaction. In: Proceedings of the Fifth Annual ESPRIT Conference, ESPRIT’ 88, Putting the Technology to Use, Brussels, Belgium, November, 1988, pp 562-577Google Scholar
  4. [Bolt80]
    Bolt RA. “Put-That-There”: Voice and Gesture at the Graphics Interface. In: Proceedings of SIGGRAPH Conference, Seattle, Washington, July, 1980, pp 262-270Google Scholar
  5. [Bolt87]
    Bolt RA. Conversing With Computers. In: Readings in Human Computer Interaction, Ch. 14: Research Frontiers and Unsolved Problems, Baecker R, Buxton W (eds), Morgan Kaufmann, California, 1987, pp 694–702Google Scholar
  6. [Brow79]
    Brown DC, Kwasny SC, Chandrasekaran B, Sondeheimer NK. An Experimental Graphics System with Natural Language Input, Computer and Graphics, 1979; 4: 13–22CrossRefGoogle Scholar
  7. [Carr87]
    Carroll JM. The Adventure of Getting to Know a Computer. In: Readings in Human Computer Interaction, Baecker RM, William AS. (eds), Morgan Kaufmann, 1987, pp 639-648Google Scholar
  8. [Fähn84]
    Fähnrich K-P, Hanne K-H, Hoepelman JPh, Zeigler J. A Logic Oriented Approach to Knowledge and Databases Supporting Natural User Interaction — LOKI, Delivery Report E3: “The Role of Graphics”, Task GR/IAO, Fraunhofer-Institut für Arbeitswirtschaft und Organisation, 1984Google Scholar
  9. [Habe81]
    Haber RN. The Power of Visual Perceiving. Journal of Mental Imagery, 1981; 5: 1–40MathSciNetGoogle Scholar
  10. [HaHo88]
    Hanne K-H, Hoepelman JPh. Man-Computer Interaction Combining Graphic and Natural Language Interaction. In: Information Technology for Organisational Systems, Bullinger H-J. et al (eds), Elsevier Science, Brussels, 1988, pp 410–415Google Scholar
  11. [Hayc86]
    Hayes PJ. Steps Towards Integrating Natural Language and Graphical Interaction for Knowledge Based Systems, Proceedings of the 7th European Conference on Artificial Intelligence, ECAI’ 86, Advances in Artificial Intelligence H, Brighton, UK, July, 1986, pp 543-552Google Scholar
  12. [Hcnd77]
    Hendrix GG. Human engineering for applied natural language processing. In: UCAI-77: proceedings of the 5th International Joint Conference on Artificial Intelligence, 1977, pp. 183-191.Google Scholar
  13. [Howe91]
    Howes M, Ghali N, Sheeny N, Mardia K, Wilson MD. ESPRIT Project 2474, A Multi-Modal Interface for Man Machine Interaction with Knowledge Based Systems, d32, Gesture Mode, 1991Google Scholar
  14. [Kobs86]
    Kobsa A, Allgayer J, Reddig C et al. Combining Deictic Gestures and Natural Language for Referent Identification. In: Procecdings of the 11th International Conference on Computational Linguistics — COLING’ 86, Bonn, Germany, August, 1986, pp. 356-361Google Scholar
  15. [LeKM89]
    Lee J, Kemp B, Manz T. Knowledge — Based Graphical Dialogue: A Strategy and Architecture. In: Proceedings of the ESPRIT’ 89 Conference, Brussels, Belgium, November-December, 1989, pp. 321-333.Google Scholar
  16. [LeZe84]
    Lee JR, Zeevat H., Integrating Natural Language and Graphics in Dialogue. In: Human Computer Interaction: Proceedings of INTERACT’ 84 Conference, London, UK, September, 1984, pp 479-484Google Scholar
  17. [Malo82]
    Malone TW. Heuristics for Designing Enjoyable User Interfaces: Lessons From Computer Games. In: Proceedings of Human Factors in Computer Systems Conference, Gaithersburg, MD, March, 1982Google Scholar
  18. [Neal88]
    Neal JG et al. Multi-Modal References in Human Computer Dialogue, Proceedings of the AAAI’ 88 Conference, Michigan, USA, June,, 1988 pp. 819-823.Google Scholar
  19. [Pine89]
    Pineda LA. GRAFLOG: A Theory of Semantics for Graphics with Applications to Human-Computer Interaction and CAD Systems, Unpublished PhD Thesis, University of Edinburgh, 1989Google Scholar
  20. [Sali90]
    Salisbury MW, Hendrickson JH, Lammers TL, Fu C, Moody SA. Talk and Draw: Bundling Speech and Graphics, Computer, 1990; 23: 8: 59–65CrossRefGoogle Scholar
  21. [Schm90]
    Schmauks D, Reithinger N. Generating Multimodal Output — Conditions, Advantages and Problems. In: Proceedings of 12th International Conference on Computational Linguistics — COLING’ 88, Vol II, Budapest, August, 1988, pp 584-588Google Scholar
  22. [Schn82]
    Schneiderman B. The Future of Interactive Systems and the Emergence of Direct Manipulation, Journal of Behaviour and Information Technology, 1982; 1,: 237–256.Google Scholar
  23. [Simp88]
    Simpkins NK. A Unification-Based Natural Language Interface to a Database, Unpublished PhD Thesis, University of Aston in Birmingham, 1988Google Scholar
  24. [Suth90]
    Sutherland E. MINITEL-The Resistible Rise of French Videotex, International Journal of Information Resource Management 1990; 1: 4: 4–14Google Scholar
  25. [UdOh82]
    Udagawa Y, Ohsuga S. Novel Techniques to Interact with Relational Databases by Using a Graphics Display, Journal of Information Processing 1982; 5: 4: 256–264Google Scholar
  26. [Zing90]
    Zinger JL, Sedlock D, Perez R, Doe GJ, Wilson MD. First Prototype of the Integrated Dialog Management, Deliverable d5, ESPRIT Project 2474 MMI2, A Multi-Modal Interface for Man Machine Interaction with Knowledge Based Systems, 1990Google Scholar

Copyright information

© British Computer Society 1993

Authors and Affiliations

  • Jane Littlehales
    • 1
  • Peter Hancox
    • 1
  1. 1.School of Computer ScienceUniversity of BirminghamUK

Personalised recommendations