Knowledge-based Generation of Illustrated Documents

  • Wolfgang Wahlster
  • Elisabeth André
  • Som Bandyopadhyay
  • Winfried Graf
  • Thomas Rist
Conference paper
Part of the Informatik-Fachberichte book series (INFORMATIK, volume 287)


The task of the knowledge-based presentation system WIP is the generation of a variety of multimodal documents from an input consisting of a formal description of the communicative intent of a planned presentation. WIP generates illustrated texts that are customized for the intended audience and situation. We present the architecture of WIP and introduce as its major components the presentation planner, the layout manager, the text generator and the graphics generator. An extended notion of coherence for multimodal documents is introduced that can be used to constrain the presentation planning process. The paper focuses on the coordination of contents planning and layout that is necessary to produce a coherent illustrated text. In particular, we discuss layout revisions after contents planning and the influence of layout constraints on text generation. We show that in WIP the design of a multimodal document is viewed as a non-monotonic planning process that includes various revisions of preliminary results in order to achieve a coherent output with an optimal media mix.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [Allgayer et al. 89]
    Allgayer, J., Harbusch, K., Kobsa, A., Reddig, C., Reithiriger, N., and Schmauks, D.: XTRA: A Natural Language Access System to Expert Systems. In: International Journal of Man-Machine Studies Vol. 31 (1989) 161–195CrossRefGoogle Scholar
  2. [André&Rist 90a]
    André, E. and Rist, T.: Towards a Plan-Based Synthesis of Illustrated Documents. In: Proc. of the 9th European Conference on Artificial Intelligence (1990) 25–30Google Scholar
  3. [André&Rist 90b]
    André, E. and Rist, T.: Synthesizing Illustrated Documents: A Plan- Based Approach. In: Proc. of Info Japan 90, Vol. 2 (1990) 163–170Google Scholar
  4. [Bandyopadhyay 90]
    Bandyopadhyay, S.: Towards an Understanding of Coherence in Multimodal Discourse. Technical Memo DFKI-TM-90-01, German Research Center for Artificial Intelligence, Saarbrücken Site (1990)Google Scholar
  5. [Beach 85]
    Beach, R. J.: Setting Tables and Illustrations with Style. Xerox PARC Technical Report CSL-85–3 (1985)Google Scholar
  6. [Borning&Duisberg 86]
    Borning, A. and Duisberg, A.: Constraint-Based Tools for Building User Interfaces. ACM Trans, on Graphics 5:6 (1986) 345–374CrossRefGoogle Scholar
  7. [Borning et al. 89]
    Borning, A., Freeman-Benson, B., and Wilson, M.: Constraint Hierarchies. Internal Report, Department of Computer Science and Engineering, FR- 35, University of Washington, Seattle (1989)Google Scholar
  8. [Feiner 88]
    Feiner, S.: A Grid-Based Approach to Automating Display Layout. In: Proc, of the Graphics Interface 88. Palo Alto: Morgan Kaufmann (1988) 192- 197Google Scholar
  9. [Feiner&McKeown 89]
    Feiner, S. and McKeown, K.: Coordinating Text and Graphics in Explanation Generation. In: DARPA Speech and Natural Language Workshop (1989)Google Scholar
  10. [Finkler&Neumann 89]
    Finkler, W. and Neumann, G.: POPEL-HOW: A Distributed Parallel Model for Incremental Natural Language Production with Feedback. In: Proc. of the 11th IJCAI (1989) 1518–1523Google Scholar
  11. [Freeman-Benson et al. 90]
    Freeman-Benson, B., Maloney, J., and Borning, A.: An Incremental Constraint Solver. Communications of the ACM, Vol. 33, No. 1 (1990) 54–63CrossRefGoogle Scholar
  12. [Graf&Maaß 91]
    Graf, W. and Maaß, W.: Constraint-basierte Verarbeitung graphischen Wissens. To appear in Proc. of the 4th Int. GI-Kongreß Wissensbasierte Systeme — Verteilte KI. Berlin: Springer-Verlag (1991)Google Scholar
  13. [Harbusch 90]
    Harbusch, K.: Constraining Tree Adjoining Grammars by Unification. Proc. of the 13th COLING (1990) 167–172Google Scholar
  14. [Jameson&Wahlster 82]
    Jameson, A. and Wahlster, W.: User Modelling in Anaphora Generation: Ellipsis and Definite Description. In: Proc. of the 5th ECAI (1982) 222–227Google Scholar
  15. [Kjorup 78]
    Kjorup, S.: Pictorial Speech Acts. In: Erkenntnis 12 (1978) 55–71CrossRefGoogle Scholar
  16. [Mann&Thompson 88]
    Mann, W. and Thompson, S.: Rhetorical Structure Theory: Towards a Functional Theory of Text Organization. In: TEXT, 8 (3) (1988)Google Scholar
  17. [Marks&Reiter 90]
    Marks, J. and Reiter, E.: Avoiding Unwanted Conversational Im- plicatures in Text and Graphics. In: Proc. of the 8th AAAI (1990) 450–455Google Scholar
  18. [Maaß 91]
    Maaß, W.: Constraint-basierte Repräsentation von graphischem Wissen am Beispiel des Layout-Managers in WIP. MS thesis, Computer Science Departement, University of Saarbrücken (1991)Google Scholar
  19. [McKeown&Feiner 90]
    McKeown, K. and Feiner, S.: Interactive Multimedia Explanation for Equipment Maintenance and Repair. In: DARPA Speech and Natural Language Workshop (1990) 42–47Google Scholar
  20. [Moore&Paris 89]
    Moore, J. and Paris, C.: Planning Text for Advisory Dialogues. In: Proc. of the 27th ACL (1989) 203–211Google Scholar
  21. [Müller-Brockmann 81]
    Müller-Brockmann, J.: Grid Systems in Graphic Design. Stuttgart: Hatje (1981)Google Scholar
  22. [Neal&Shapiro 88]
    Neal, J. and Shapiro, S.: Intelligent Multi-Media Interface Technology. In: Proc. of the Workshop on Architectures of intelligent Interfaces: Ele- ments&Prototypes (1988) 69–91Google Scholar
  23. [Nebel 90]
    Nebel, B.: Reasoning and Revision in Hybrid Representation Systems. Lecture Notes in AI, Vol. 422, Berlin: Springer-Verlag (1990)Google Scholar
  24. [Rist&André 90]
    Rist, T. and André, E.: Wissensbasierte Perspektivenwahl für die automatische Erzeugung von 3D-Objektdarstellungen. In: K. Kansy and P. Wißkirchen (eds.). Graphik und KI. IFB 239, Berlin: Springer-Verlag (1990) 48- 57CrossRefGoogle Scholar
  25. [Roth et al. 88]
    Roth, S., Mattis, J., and Mesnard, X.: Graphics and Natural Language as Components of Automatic Explanation. In: Proc. of the Workshop on Architectures of Intelligent Interfaces: Elements & Prototypes (1988) 109–128Google Scholar
  26. [Searle 69]
    Searle, J.: Speech Acts: An Essay in the Philosophy of Language. Cambridge, MA: Cambridge University Press (1969)Google Scholar
  27. [Schauder 90]
    Schauder, A.: Inkrementelle syntaktische Generierung natürlicher Sprache mit Tree Adjoining Grammars. MS thesis, Computer Science Departement, University of Saarbrücken (1990)Google Scholar
  28. [Stock 91]
    Stock, O.: Natural Language and Exploration of an Information Space: the ALFresco Interactive System. Technical Report, Istituto per la Ricerca Scientifica e Tecnologica, Trento, Italy (1991)Google Scholar
  29. [Van Dijk 80]
    van Dijk, T.: Textwissenschaft. München: dtv (1980)CrossRefGoogle Scholar
  30. [Wahlster 91]
    Wahlster, W.: User and Discourse Models for Multimodal Communication. In: J. Sullivan and S. Tyler (eds.). Architectures for Intelligent User Interfaces: Elements and Prototypes. Reading, MA: Addison-Wesley (1991)Google Scholar
  31. [Wahlster&Kobsa 89]
    Wahlster, W. and Kobsa, A.: User Models in Dialog Systems. In: A. Kobsa and W. Wahlster (eds.). User Models in Dialog Systems. Symbolic Computation Series, Berlin: Springer-Verlag (1989) 4–34CrossRefGoogle Scholar
  32. [Wahlster et al. 89]
    Wahlster, W., André, E., Hecking, M., and Rist, T.: WIP: Knowledge-based Presentation of Information. Report WIP-1, German Research Center for Artificial Intelligence, Saarbrücken (1989)Google Scholar
  33. [Wahlster et al. 91]
    Wahlster, W., André, E., Graf, W., and Rist, T.: Designing Illustrated Texts: How Language Production Is Influenced by Graphics Generation. In: Proc. of the 5th Conference of the European Chapter of the ACL, Berlin: Springer-Verlag (1991) 8–14Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1991

Authors and Affiliations

  • Wolfgang Wahlster
    • 1
  • Elisabeth André
    • 1
  • Som Bandyopadhyay
    • 1
  • Winfried Graf
    • 1
  • Thomas Rist
    • 1
  1. 1.German Research Center for Artificial Intelligence (DFKI)Saarbrücken 11Germany

Personalised recommendations