Skip to main content
SpringerLink
Log in
Book cover

Multimodal Interaction with W3C Standards pp 109–135Cite as

Dialogue Act Annotation with the ISO 24617-2 Standard

  • Chapter
  • First Online:

Abstract

This chapter describes recent and ongoing annotation efforts using the ISO 24617-2 standard for dialogue act annotation. Experimental studies are reported on the annotation by human annotators and by annotation machines of some of the specific features of the ISO annotation scheme, such as its multidimensional annotation of communicative functions, the recognition of each of its nine dimensions, and the recognition of dialogue act qualifiers for certainty, conditionality, and sentiment. The construction of corpora of dialogues, annotated according to ISO 24617-2, is discussed, including the recent DBOX and DialogBank corpora.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    See Allen and Core [2], Dhillon et al. [19], Carletta et al. [16], Jurafsky et al. [32], Alexandersson et al. [1], Bunt [4, 5].

  2. 2.

    DIT\( {}^{++} \) has a fine-grained set of 29 feedback functions, whereas ISO 241617-2 has only 5, which are, however, more reliably annotated.

  3. 3.

    ISO 24617-2 does not prescribe the use of any particular set of sentiment labels. See, e.g., the EmotionML language (www.w3.org/TR/emotionml) for possible choices in this respect.

  4. 4.

    See Geertzen et al. [25].

  5. 5.

    See http://www.let.rug.nl/vannoord/Ovis/.

  6. 6.

    See https://www.cs.rochester.edu/research/speech.

  7. 7.

    Eureka project E! 7152, see https://www.lsv.uni-saarland.de/index.php?id=71.

  8. 8.

    See http://www.metalogue.eu.

  9. 9.

    See, for example, http://www.youtube.com/watch?v=g2Fg-LJHPA4. For information about the UK Youth Parliament, see http://www.ukyouthparliament.org.uk/

  10. 10.

    The Switchboard corpus is distributed by the Linguistic Data Consortium: https://www.ldc.upenn.edu.

  11. 11.

    The remaining 10.29 % of SWBD-DAMSL tags cannot be converted into ISO tags since they are not really concerned with communicative functions, such as the SWBD-DAMSL tags ‘non-verbal’, ‘uninterpretable’, ‘quoted material’, ‘transcription error’.

  12. 12.

    See http://doc//.ukdataservice.ac.uk/doc/4632/mrdoc/pdf/4632userguide.pdf.

  13. 13.

    See Prüst et al. [42].

  14. 14.

    The Switchboard corpus is also available in NXT format [15], without in-line markups.

  15. 15.

    Text Encoding Initiative: www.tei.org.

References

  1. Alexandersson, J., Buschbeck-Wolf, B., Fujinami, T., Kipp, M., Koch, S., Maier, E., et al. (1998). Dialogue acts in VERBMOBIL-2 (second edition). Verbmobil Report 226. Saarbrücken: DFKI.

    Google Scholar 

  2. Allen, J., & Core, M. (1997). DAMSL: Dialogue act markup in several layers (Draft 2.1). Technical Report. Rochester, NY: University of Rochester.

    Google Scholar 

  3. Allwood, J. (1992). On dialogue cohesion. Gothenburg University, Department of Linguistics.

    Google Scholar 

  4. Bunt, H. (1994). Context and dialogue control. Think Quarterly, 3(1), 19–31.

    Google Scholar 

  5. Bunt, H. (2000). Dialogue pragmatics and context specification. In H. Bunt & W. Black (Eds.), Abduction, belief and context in dialogue. Studies in computational pragmatics (pp. 81–150). Amsterdam: John Benjamins.

    Google Scholar 

  6. Bunt, H. (2006). Dimensions in dialogue annotation. In Proceedings 5th International Conference on Language Resources and Evaluation (LREC 2006), Genova, Paris. ELRA.

    Google Scholar 

  7. Bunt, H. (2009). The DIT\( {}^{++} \) taxonomy for functional for dialogue markup. In D. Heylen, C. Pelachaud, R. Catizone, & D. Traum (Eds.), Proceedings of EDAML-AAMAS Workshop “Towards a Standard Markup Language for Embodied Dialogue Acts”, Budapest (pp. 36–48).

    Google Scholar 

  8. Bunt, H. (2011). Multifunctionality in dialogue. Computer, Speech and Language, 25, 222–245.

    Article  Google Scholar 

  9. Bunt, H. (2015). On the principles of semantic annotation. In Proceedings 11th Joint ACL-ISO Workshop on Interoperable Semantic Annotation (ISA-11), London (pp. 1–13).

    Google Scholar 

  10. Bunt, H., Alexandersson, J., Carletta, J., Choe, J.-W., Fang, A., Hasida, K., et al. (2010). Towards and ISO standard for dialogue act annotation. In Proceedings 7th International Conference on Language Resources and Evaluation (LREC 2010), Malta, Paris. ELDA.

    Google Scholar 

  11. Bunt, H., Alexandersson, J., Choe, J.-W., Fang, A., Hasida, K., Petukhova, V., et al. (2012). ISO 24617-2: A semantically-based standard for dialogue annotation. In Proceedings of 8th International Conference on Language Resources and Evaluation (LREC 2012), Istanbul. Paris: ELDA.

    Google Scholar 

  12. Bunt, H., Fang, A., Cao, J., Liu, X., & Petukhova, V. (2013). Issues in the addition of ISO standard annotations to the Switchboard corpus. In Proceedings 9th Joint ISO - ACL SIGSEM Workshop on Interoperable Semantic Annotation (ISA-9), Potsdam (pp. 59–70).

    Google Scholar 

  13. Bunt, H., Kipp, M., & Petukhova, V. (2012). Using DiAML and ANVIL for multimodal dialogue annotation. In Proceedings 8th International Conference on Language Resources and Evaluation (LREC 2012), Istanbul. Paris: ELRA.

    Google Scholar 

  14. Bunt, H., Petukhova, V., Malchanau, A., & Wijnhoven, K. (2016). The DialogBank. In Proceedings 10th International Conference on Language Resources and Evaluation (LREC 2016), Portoroz, Slovenia. Paris: ELRA.

    Google Scholar 

  15. Calhoun, S., Carletta, J., Brenier, J., Mayo, N., Jurafsky, D., Steedman, M., et al. (2010). The NXT-format Switchboard corpus: A rich resource for investigating the syntax, semantics, pragmatics and prosody of dialogue. Language Resources and Evaluation, 44(4), 387–419.

    Article  Google Scholar 

  16. Carletta, J., Isard, S., Kowtko, J., & Doherty-Sneddon, G. (1996). HCRC dialogue structure coding manual. Technical Report HCRC/TR-82, University of Edinburgh.

    Google Scholar 

  17. Cohen, J. (1960). A coefficient of agreement for nominal scales. Education and Psychological Measurement, 20, 37–46.

    Article  Google Scholar 

  18. Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213–261.

    Article  Google Scholar 

  19. Dhillon, R., Bhagat, S., Carvey, H., & Schriberg, E. (2004). Meeting recorder project: Dialogue labelling guide. ICSI Technical Report TR-04-002. University of California at Berkeley.

    Google Scholar 

  20. Fang, A., Cao, J., Bunt, H., & Liu, X. (2011). Relating the semantics of dialogue acts to linguistic properties: A machine learning perspective through lexical cues. In Proceedings IEEE-ICSC 2011 Workshop on Semantic Annotation for Computational Linguistic Resources, Stanford, CA.

    Google Scholar 

  21. Fang, A., Cao, J., Bunt, H., & Liu, X. (2012). The annotation of the Switchboard corpus with the new ISO standard for dialogue act analysis. In Proceedings 8th Joint ISO - ACL SIGSEM Workshop on Interoperable Semantic Annotation (ISA-8), Pisa (pp. 13–18).

    Google Scholar 

  22. Fang, A., Cao, J., Bunt, H., & Liu, X. (2012). Applicability verification of a new ISO standard for dialogue act annotation with the Switchboard corpus. In Proceedings of EACL 2012 Workshop on Innovative Hybrid Approaches to the Processing of Textual Data, Avignon.

    Google Scholar 

  23. Geertzen, J. (2007). DitAT: A flexible tool to support web-based dialogue annotation. In Proceedings 7th International Conference on Computational Semantics (IWCS-7), Tilburg (pp. 320–323).

    Google Scholar 

  24. Geertzen, J., & Bunt, H. (2006). Measuring annotator agreement in a complex, hierarchical dialogue act schema. In Proceedings SIGDIAL 2006, Sydney.

    Google Scholar 

  25. Geertzen, J., Girard, Y., & Morante, R. (2004). The DIAMOND project. In Proceedings of 8th Workshop on the Semantics and Pragmatics of Dialogue (CATALOG 2004), Barcelona.

    Google Scholar 

  26. Geertzen, J., Petukhova, V., & Bunt, H. (2008). Evaluating dialogue act tagging with naive and expert annotators. In Proceedings 6th International Conference on Language Resources and Evaluation (LREC 2008), Marrakech. Paris: ELDA.

    Google Scholar 

  27. Hovy, E., & Maier, E. (1995). Parsimonious or profligate: How many and which discourse structure relations? ISI Research Report. Marina del Rey: Information Sciences Institute, University of Southern California.

    Google Scholar 

  28. Ide, N., & Romary, L. (2004). International standard for a linguistic annotation framework. Natural Language Engineering, 10, 211–225.

    Article  Google Scholar 

  29. ISO (2011). ISO 24612: Language Resource Management - Linguistic Annotation Framework (LAF). Geneva: ISO.

    Google Scholar 

  30. ISO (2012). ISO 24617-2: Language Resource Management - Semantic Annotation Framework (SemAF) - Part 2: Dialogue Acts. Geneva: ISO.

    Google Scholar 

  31. ISO (2016). ISO 24617-6: Language Resource Management - Semantic Annotation Framework (SemAF) - Part 6: Principles of Semantic Annotation. Geneva: ISO.

    Google Scholar 

  32. Jurafsky, D., Shriberg, E., & Biasca, D. (1997). Switchboard SWBD-DAMSL shallow-discourse-function annotation: Coders manual, Draft 1.3. University of Colorado.

    Google Scholar 

  33. Lendvai, P., van den Bosch, A., Krahmer, E., & Canisius, S. (2004). Memory-based robust interpretation of recognised speech. In Proceedings 9th International Conference on Speech and Computer (SPECOM’04), St. Petersburg (pp. 415–422).

    Google Scholar 

  34. Lesch, S., Kleinbauer, T., & Alexandersson, J. (2005). A new metric for the evaluation of dialog act classification. In Proceedings 9th Workshop on the Semantics and Pragmatics of Dialogue (DIALOR), Nancy.

    Google Scholar 

  35. Meteer, M., & Taylor, A. (1995). Dysflency annotation stylebook for the Switchboard corpus. Washington: Linguistic Data Consortium.

    Google Scholar 

  36. Petukhova, V. (2011). Multidimensional dialogue modelling. Ph.D. dissertation. Tilburg University.

    Google Scholar 

  37. Petukhova, V., & Bunt, H. (2011). Incremental dialogue act understanding. In Proceedings Ninth International Conference on Computational Semantics (IWCS 2011), Oxford (pp. 235–244).

    Google Scholar 

  38. Petukhova, V., Gropp, M., Klakow, D., Eigner, G., Topf, M., Srb, S., et al. (2014). The DBOX corpus collection of spoken human-human and human-machine dialogues. In Proceedings 9th International Conference on Language Resources and Evaluation (LREC 2014), Reykjavik, Iceland.

    Google Scholar 

  39. Petukhova, V., Prévot, L., & Bunt, H. (2011). Multi-level discourse relations between dialogue units. In Proceedings 6th Joint ACL-ISO Workshop on Interoperable Semantic Annotation (ISA-6), Oxford (pp. 18–28).

    Google Scholar 

  40. Popescu-Belis, A. (2005). Dialogue acts: One or more dimensions? ISSCO Working Paper 62. Geneva: ISSCO.

    Google Scholar 

  41. Prasad, R., Dinesh, N., Lee, A., Miltsakaki, E., Robaldo, L., Joshi, A., et al. (2008). The Penn Discourse TreeBank 2.0. In Proceedings 6th International Conference on Language Resources and Systems (LREC 2008), Marrakech.

    Google Scholar 

  42. Prüst, H., Minnen, G., & Beun, R.-J. (1984). Transcriptie dialooogesperiment juni/juli 1984. IPO Rapport 481. Institute for Perception Research, Eindhoven University of Technology.

    Google Scholar 

  43. Reithinger, N., & Klesen, M. (1997). Dialogue act classification using language models. In Proceedings of Eurospeech-97 (pp. 2235–2238).

    Google Scholar 

  44. Samuel, K., Carberry, S., & Vijay-Shanker, K. (1998). Dialogue act tagging with transformation-based learning. In Proceedings ACL 1998, Montreal (pp. 1150–1156).

    Google Scholar 

  45. Stolcke, A., Res, K., Coccaro, K., Shriberg, E., Bates, R., Jurafsky, D., et al. (2000). Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3), 339–373.

    Article  Google Scholar 

  46. Traum, D. (2000). 20 questions on dialogue act taxonomies. Journal of Semantics, 17(1), 7–30.

    Article  Google Scholar 

  47. Zimmermann, M., Lui, Y., Shriberg, E., & Stolcke, A. (2005). Toward joint segmentation and classification of dialogue acts in multiparty meetings. In Proceedings of the Multimodal Interaction and Related Machine Learning Algorithms Workshop (MLMI-05) (pp. 187–193). Berlin: Springer.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tilburg Center for Cognition and Communication (TiCC), Tilburg University, Tilburg, The Netherlands

    Harry Bunt

  2. Spoken Language Systems Group, Saarland University, Saarbrücken, Germany

    Volha Petukhova

  3. Institute fro Creative Technologies, University of Southern California, Los Angeles, CA, 90094, USA

    David Traum

  4. Department of Intelligent User Interfaces DFKI, Gernan Research Center for Artificial Intelligence, Saarbrücken, Germany

    Jan Alexandersson

Authors
  1. Harry Bunt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Volha Petukhova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Traum
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jan Alexandersson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Harry Bunt .

Editor information

Editors and Affiliations

  1. Conversational Technologies, Plymouth Meeting, Pennsylvania, USA

    Deborah A. Dahl

Appendix

Appendix

This appendix shows the ISO 24617-2 annotation of the first two utterances of a Map Task dialogue in the DialogBank corpus, as produced with the ANVIL tool and exported in DiAML format. In a TEI-compliant way,Footnote 15 the first part identifies the two dialogue participants (“p1” and “p2”), followed by a second part that identifies the word tokens in the audio-video input stream, and a third part that identifies the functional segments in terms of the word tokens. The last part represents the dialogue act annotations in the DIAML format of the ISO standard.

  1. (14)

    G: right

    G: go south and you’ll pass some cliffs on your right

    F: okay

    <?xml version="1.0" encoding="UTF-8"?>

    <TEI xmlns:="http://www.tei-c.org/ns/1.0">

      <profileDescr xmlns:="">

        <particDescr xml:id="p1">

          <p>the 1. participant</p>

        </particDescr>

        <particDescr xml:id="p2">

          <p>the 2. participant</p>

        </particDescr>

      </profileDescr>

      <text>

        <body />

        <div>

          <head>The dialogue turns, segmented into words

         (TEI-compliant)</head>

          <u>

          <w xml:id="w1">right</w>

            <w xml:id="w2">go</w>

            <w xml:id="w3">south</w>

            <w xml:id="w4">and</w>

            <w xml:id="w5">you’ll</w>

            <w xml:id="w6">pass</w>

            <w xml:id="w7">some</w>

            <w xml:id="w8">cliffs</w>

            <w xml:id="w9">on</w>

            <w xml:id="w10">your</w>

            <w xml:id="w11">right</w>

            <w xml:id="w12">okay</w>

            ...

            </u>

          </div>

        <div>

          <head>Identification of functional segments</head>

          <spanGrp xml:id="ves1" type="functionalVerbalSegment">

            <span xml:id="ts1" type="textStretch" from="w1" to="w1" />

          </spanGrp>

          <fs type="functionalSegment" xml:id="fs1">

            <f name="verbalComponent" fVal="#ves1" />

          </fs>

          <spanGrp xml:id="ves2" type="functionalVerbalSegment">

            <span xml:id="ts2" type="textStretch" from="w2" to="w11" />

          </spanGrp>

         <fs type="functionalSegment" xml:id="fs2">

            <f name="verbalComponent" fVal="#ves2" />

          </fs>

          <spanGrp xml:id="ves3" type="functionalVerbalSegment">

            <span xml:id="ts3" type="textStretch" from="w12" to="w12" />

          </spanGrp>

         <fs type="functionalSegment" xml:id="fs3">

            <f name="verbalComponent" fVal="#ves3" />

          </fs>

        </div>

        <diaml xmlns:="http://www.iso.org/diaml">

        <dialogueAct xml:id="da1"

        target="#fs1" sender="#p1" addressee="#p2"

        dimension="turnManagement" communicativeFunction="turnTake"  />

        <dialogueAct xml:id="da2"

        target="#fs1" sender="#p1" addressee="#p2"

        dimension="discourseStructuring" communicativeFunction="opening"  />

        <dialogueAct xml:id="da3"

        target="#fs2" sender="#p1" addressee="#p2"

        dimension="task" communicativeFunction="instruct"  />

        <dialogueAct xml:id="da4"

        target="#fs3" sender="#p2" addressee="#p1"

        dimension="autoFeedback" communicativeFunction="autoPositive"

        feedbackDependence="#fs2" />

        </diaml>

     </text>

    </TEI>

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Bunt, H., Petukhova, V., Traum, D., Alexandersson, J. (2017). Dialogue Act Annotation with the ISO 24617-2 Standard. In: Dahl, D. (eds) Multimodal Interaction with W3C Standards. Springer, Cham. https://doi.org/10.1007/978-3-319-42816-1_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-42816-1_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-42814-7

  • Online ISBN: 978-3-319-42816-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation