A Multimodal Discourse Ontology for Meeting Understanding

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3869)


In this paper, we present a multimodal discourse ontology that serves as a knowledge representation and annotation framework for the discourse understanding component of an artificial personal office assistant. The ontology models components of natural language, multimodal communication, multi-party dialogue structure, meeting structure, and the physical and temporal aspects of human communication. We compare our models to those from the research literature and from similar applications. We also highlight some annotations which have been made in conformance with the ontology as well as some algorithms which have been trained on these data and suggest elements of the ontology that may be of immediate interest for further annotation by human or automated means.


Communication Model Communicate Event Discourse Structure Speech Recognizer Annotation Framework 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Romano Jr., N.C., Nunamaker Jr., J.F.: Meeting analysis: Findings from research and practice. In: Proceedings of the 34th Hawaii International Conference on System Sciences (2001)Google Scholar
  2. 2.
    Lisowska, A., Popescu-Belis, A., Armstrong, S.: User query analysis for the specificationand evaluation of a dialogue processing and retrieval system. In: Proceedings of the 4th International Conference on Language Resources and Evaluation (2004)Google Scholar
  3. 3.
    Reidsma, D., Rienks, R., Jovanović, N.: Meeting modelling in the context of multimodal research. In: Bengio, S., Bourlard, H. (eds.) MLMI 2004. LNCS, vol. 3361, pp. 22–35. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  4. 4.
    Bachler, M.S., Shum, S.J.B., Roure, D.C.D., Michaelides, D.T., Page, K.R.: Ontologicalmediation of meeting structure: Argumentation, annotation, and navigation. In: Proceedings of the 1st International Workshop on Hypermedia and the Semantic Web (2003)Google Scholar
  5. 5.
    Marchand-Maillet, S.: Meeting record modelling for enhanced browsing. Technical Report 03.01, Computer Vision and Multimedia Laboratory, Computing Centre, University of Geneva, Switzerland (2003)Google Scholar
  6. 6.
    Banerjee, S., Rose, C., Rudnicky, A.: The necessity of a meeting recording and playback system, and the benefit of topic–level annotations to meeting browsing. In: Costabile, M.F., Paternó, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 643–656. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  7. 7.
    Barker, K., Porter, B., Clark, P.: A library of generic concepts for composing knowledge bases. In: Proceedings of the 1st International Conference on Knowledge Capture (2001)Google Scholar
  8. 8.
    Clark, P., Porter, B.: KM - The Knowledge Machine 2.0: Users manual (2004),
  9. 9.
    Popescu-Belis, A.: Dialogue acts: One or more dimensions? ISSCO Working Paper 62. University of Geneva (2005)Google Scholar
  10. 10.
    Clark, H.H., Krych, M.A.: Speaking while monitoring addressees for understanding. Journal of Memory and Language 50, 62–81 (2004)CrossRefGoogle Scholar
  11. 11.
    Quek, F., McNeill, D., Bryll, R., Duncan, S., Ma, X.F., Kirbas, C., McCullough, K.E., Ansari, R.: Multimodal human discourse: Gesture and speech. ACM Transactions on Computer-Human Interaction 9(3), 171–193 (2002)CrossRefGoogle Scholar
  12. 12.
    Farrar, S., Langendoen, T.: A linguistic ontology for the semantic web. Glot International 7(3), 97–100 (2003)Google Scholar
  13. 13.
    Ide, N., Romary, L., de la Clergerie, E.: International standard for a linguistic annotation framework. In: Proceedings of the HLT-NAACL Workshop on the Software Engineering and Architecture of Language Technology (2003)Google Scholar
  14. 14.
    Clark, A., Popescu-Belis, A.: Multi-level dialogue act tags. In: Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue (2004)Google Scholar
  15. 15.
    Shriberg, E., Dhillon, R., Bhagat, S., Ang, J., Carvey, H.: The ICSI Meeting Recorder Dialog Act Corpus. In: Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue (2004)Google Scholar
  16. 16.
    Lemon, O., Gruenstein, A.: Multithreaded context for robust conversational interfaces: Context-sensitive speech recognition and interpretation of corrective fragments. ACM Transactions on Computer-Human Interaction 11(3) (2004)Google Scholar
  17. 17.
    Traum, D., Bos, J., Cooper, R., Larsson, S., Lewin, I., Matheson, C., Poesio, M.: A model of dialogue moves and information state revision. Task Oriented Instructional Dialogue (TRINDI): Deliverable 2.1. University of Gothenburg (1999)Google Scholar
  18. 18.
    Pallotta, V., Niekrasz, J., Purver, M.: Collaborative and argumentative models of natural discussions. In: Proceedings of the 5th Workshop on Computational Models of Natural Argument (2005)Google Scholar
  19. 19.
    Dielmann, A., Renals, S.: Dynamic bayesian networks for meeting structuring. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (2004)Google Scholar
  20. 20.
    Reiter, S., Rigoll, G.: Segmentation and classification of meeting events using multiple classifier fusion and dynamic programming. In: Proceedings of the International Conference on Pattern Recognition (2004)Google Scholar
  21. 21.
    McCowan, I., Gatica-Perez, D., Bengio, S., Lathoud, G., Barnard, M., Zhang, S.: Automatic analysis of multimodal group actions in meetings. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(3), 305–317 (2005)CrossRefGoogle Scholar
  22. 22.
    McCowan, I., Bengio, S., Gatica-Perez, D., Lathoud, G., Monay, F., Moore, D., Wellner, P., Bourlard, H.: Modeling human interaction in meetings. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (2003)Google Scholar
  23. 23.
    Banerjee, S., Rudnicky, A.: Using simple speech-based features to detect the state of a meeting and the roles of the meeting participants. In: Proceedings of the 8th International Conference on Spoken Language Processing (2004)Google Scholar
  24. 24.
    Galley, M., McKeown, K., Fosler-Lussier, E., Jing, H.: Discourse segmentation of multi-party conversation. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (2003)Google Scholar
  25. 25.
    Gruenstein, A., Niekrasz, J., Purver, M.: Meeting structure annotation: Data and tools. In: Proceedings of the 6th SIGdial Workshop on Discourse and Dialogue, Lisbon, Portugal (2005)Google Scholar
  26. 26.
    Niekrasz, J., Purver, M., Dowding, J., Peters, S.: Ontology-based discourse understanding for a persistent meeting assistant. In: Proceedings of the AAAI Spring Symposium Workshop on Persistent Assistants: Living and Working with AI (2005)Google Scholar
  27. 27.
    Blei, D., Moreno, P.: Topic segmentation with an aspect hidden Markov model. In: Proceedings of the 24th Annual International Conference on Research and Development in Information Retrieval, pp. 343–348 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  1. 1.Center for the Study of Language and InformationStanford UniversityStanfordUSA

Personalised recommendations