Skip to main content
SpringerLink
Log in
Book cover

Handbook of Linguistic Annotation pp 813–832Cite as

MPQA Opinion Corpus

  • Chapter
  • First Online:

Abstract

The MPQA Opinion Corpus is a collection of documents with expression-level, multi-attribute annotations of opinions, sentiments, and other private states. This chapter describes the MPQA annotation scheme and the development of the MPQA Corpus.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   349.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   449.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   449.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    This term has been borrowed and adapted from literary theory [5].

  2. 2.

    We use the term speech event to refer to any event of speaking or writing.

  3. 3.

    Although not included on attitude and target frames, the source of these annotations can be retrieved by following the attitude and target links back to the direct subjective frames.

  4. 4.

    https://gate.ac.uk/.

  5. 5.

    Annotating all types was always possible, but having them visibly listed and clickable from the start made the task more straightforward.

  6. 6.

    See the documentation accompanying the MPQA Corpus release for specifics on the MPQA annotation file format and the directory structure for the corpus.

  7. 7.

    Insubstantial private state and speech events also include those that are not significant in the discourse.

  8. 8.

    The underlined “implicit” represents the text anchor for frames for the writer of the sentence.

  9. 9.

    Called systems in systemic functional linguistics.

References

  1. Abdul-Mageed, M., Diab, M.T., Korayem, M.: Subjectivity and sentiment analysis of modern standard Arabic. In: Proceeding of the 49th Annual Meeting of the Association for Computational Linguistics. Human Language Technologies, (Vol. 2: Short Papers), (2011)

    Google Scholar 

  2. Akkaya, C., Wiebe, J., Conrad, A., Mihalcea, R.: Improving the impact of subjectivity word sense disambiguation on contextual opinion analysis. In: Proceeding of the 15th Conference on Computational Natural Language Learning, (2011)

    Google Scholar 

  3. Andrea Esuli, F.S., Urciuoli, I.: Annotating expressions of opinion and emotion in the Italian Content Annotation Bank. In: Proceeding of the 6th International Language Resources and Evaluation, (2008)

    Google Scholar 

  4. Asher, N.: Belief in discourse representation theory. J. Philos. Logic 15, 127–189 (1986)

    Article  Google Scholar 

  5. Banfield, A.: Unspeakable Sentences. Routledge and Kegan Paul, Boston (1982)

    Google Scholar 

  6. Chatman, S.: Story and Discourse: Narrative Structure in Fiction and Film. Cornell University Press, New York (1978)

    Google Scholar 

  7. Clematide, S., Gindl, S., Klenner, M., Petrakis, S., Remus, R., Ruppenhofer, J., Waltinger, U., Wiegand, M.: Mlsa a multi-layered reference corpus for german sentiment analysis. In: Proceeding of the 8th International Conference on Language Resources and Evaluation, (2012)

    Google Scholar 

  8. Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 37–46 (1960)

    Article  Google Scholar 

  9. Cohn, D.: Transparent Minds: Narrative Modes for Representing Consciousness in Fiction. Princeton University Press, New Jersey (1978)

    Google Scholar 

  10. Cunningham, H., Maynard, D., Bontcheva, K., Tablan, V.: GATE: a framework and graphical development environment for robust NLP tools and applications. In: Proceeding of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Pennsylvania (2002)

    Google Scholar 

  11. Doležel, L.: Narrative Modes in Czech Literature. University of Toronto Press, Canada (1973)

    Google Scholar 

  12. Fauconnier, G.: Mental Spaces: Aspects of Meaning Construction in Natural Language. MIT Press, Cambridge (1985)

    Google Scholar 

  13. Fodor, J.D.: The Linguistic Description of Opaque Contexts. Outstanding dissertations in linguistics 13. Garland, New York (1979)

    Google Scholar 

  14. Ghosh, S., Tonelli, S., Johansson, R.: Mining fine-grained opinion expressions with shallow parsing. In: Proceeding of the International Conference Recent Advances in Natural Language Processing, (2013)

    Google Scholar 

  15. Halliday, M.: (1985/1994) An Introduction to Functional Grammar. London, Edward Arnold

    Google Scholar 

  16. Hermann, K.M., Blunsom, P.: The role of syntax in vector space models of compositional semantics. In: Proceeding of the 51st Annual Meeting of the Association for Computational Linguistics, (Vol. 1: Long Papers), (2013)

    Google Scholar 

  17. Johansson, R., Moschitti, A.: Relational features in fine-grained opinion analysis. Comput. Linguist. 39(3), 473–509 (2013)

    Article  Google Scholar 

  18. Kessler, J.S., Eckert, M., Clark, L., Nicolov, N.: The 2010 ICWSM JDPA Sentment Corpus for the automotive domain. In: 4th International AAAI Conference on Weblogs and Social Media Data Workshop Challenge (ICWSM-DWC 2010), (2010)

    Google Scholar 

  19. Kim, J., Li, J.J., Lee, J.H.: Evaluating multilanguage-comparability of subjectivity analysis systems. In: Proceeding of the 48th Annual Meeting of the Association for Computational Linguistics, (2010)

    Google Scholar 

  20. Kim, S.M., Hovy, E.: Determining the sentiment of opinions. In: Proceeding of the 20th International Conference on Computational Linguistics (COLING 2004), (2004)

    Google Scholar 

  21. Krippendorff, K.: Content Analysis: An Introduction to its Methodology. Sage Publications, Beverly Hills (1980)

    Google Scholar 

  22. Kuroda, S.Y.: Where epistemology, style and grammar meet: a case study from the Japanese. In: Kiparsky P., Anderson S. (eds.) A Festschrift for Morris Halle, pp. 377–391. Holt, Rinehart Winston, New York (1973)

    Google Scholar 

  23. Kuroda, S.Y.: Reflections on the foundations of narrative theory-from a linguistic point of view. In: van Dijk, T. (ed.) Pragmatics of Language and Literature, pp. 107–140. North-Holland, Amsterdam (1976)

    Google Scholar 

  24. Lan, M., Xu, Y., Niu, Z.: Leveraging synthetic discourse data via multi-task learning for implicit discourse relation recognition. In: Proceeding of the 51st Annual Meeting of the Association for Computational Linguistics, (vol. 1: Long Papers) (2013)

    Google Scholar 

  25. Lin, C., He, Y., Everson, R.: Sentence subjectivity detection with weakly-supervised learning. In: Proceeding of 5th International Joint Conference on Natural Language Processing, (2011)

    Google Scholar 

  26. Martin, J.: English Text: System and Structure. John Benjamins, Amsterdam (1992)

    Book  Google Scholar 

  27. Martin, J.: Beyond exchange: APPRAISAL systems in English. In: Hunston, S., Thompson, G. (eds.) Evaluation in Text: Authorial stance and the construction of discourse, pp. 142–175. Oxford University Press, Oxford (2000)

    Google Scholar 

  28. Meng, X., Wei, F., Liu, X., Zhou, M., Xu, G., Wang, H.: Cross-lingual mixture model for sentiment classification. In: Proceeding of the 50th Annual Meeting of the Association for Computational Linguistics, (vol. 1: Long Papers) (2012)

    Google Scholar 

  29. Mohtarami, M., Lan, M., Tan, C.L.: Probabilistic sense sentiment similarity through hidden emotions. In: Proceeding of the 51st Annual Meeting of the Association for Computational Linguistics, (vol. 1: Long Papers), (2013)

    Google Scholar 

  30. Picard, R.: Affective Computing. MIT Press, Cambridge (1997)

    Google Scholar 

  31. Quirk, R., Greenbaum, S., Leech, G., Svartvik, J.: A Comprehensive Grammar of the English Language. Longman, New York (1985)

    Google Scholar 

  32. Rapaport, W.: Logical foundations for belief representation. Cogn. Sci. 10, 371–422 (1986)

    Article  Google Scholar 

  33. Ruppenhofer, J., Rehbein, I.: Yes we can!? Annotating English modal verbs. In: Proceeding of the 8th International Conference on Language Resources and Evaluation, (2012)

    Google Scholar 

  34. Seki, Y., Evans, D.K., Ku, L.W., Sun, L., Chen, H.H., Kando, N.: Overview of multilingual opinion analysis task at NTCIR-7. In: Proceeding of NTCIR-7, (2008)

    Google Scholar 

  35. Shin, H., Kim, M., Jang, H., Cattle, A.: Annotation scheme for constructing sentiment corpus in Korean. In: Proceeding of the 26th Pacific Asia Conference on Language, Information, and Computation, (2012)

    Google Scholar 

  36. Stoyanov, V., Cardie, C., Wiebe, J.: Multi-perspective question answering using the OpQA corpus. In: Proceeding of the Human Language Technologies Conference/Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP-2005), (2005)

    Google Scholar 

  37. Taboada, M., Brooke, J., Tofiloski, M., Voll, K., Stede, M.: Lexicon-based methods for sentiment analysis. Comput. Linguist. 37(2), 1–308 (2011)

    Article  Google Scholar 

  38. Toprak, C., Jakob, N., Gurevych, I.: Sentence and expression level annotation of opinions in user-generated discourse. In: Proceeding of the 48th Annual Meeting of the Association for Computational Linguistics, (2010)

    Google Scholar 

  39. Uspensky, B.: A Poetics of Composition. University of California Press, California (1973)

    Google Scholar 

  40. Wang, S., Manning, C.: Baselines and bigrams: simple, good sentiment and topic classification. In: Proceeding of the 50th Annual Meeting of the Association for Computational Linguistics, (vol. 2: Short Papers), (2012)

    Google Scholar 

  41. White, P.: Appraisal: the language of attitudinal evaluation and intersubjective stance. In: Verschueren J., Ostman J., blommaert J., Bulcaen C. (eds.) The Handbook of Pragmatics, Amsterdam/Philadelphia: John Benjamins Publishing Company, pp 1–27. (2002)

    Google Scholar 

  42. Wiebe, J.: Recognizing subjective sentences: a computational investigation of narrative text. Ph.D. Thesis, State University of New York at Buffalo (1990)

    Google Scholar 

  43. Wiebe, J.: Tracking point of view in narrative. Comput. Linguist. 20(2), 233–287 (1994)

    Google Scholar 

  44. Wiebe, J.: Instructions for annotating opinions in newspaper articles. Department of Computer Science Technical Report TR-02-101, University of Pittsburgh (2002)

    Google Scholar 

  45. Wiebe, J., Breck, E., Buckley, C., Cardie, C., Davis, P., Fraser, B., Litman, D., Pierce, D., Riloff, E., Wilson, T., Day, D., Maybury, M.: Recognizing and organizing opinions expressed in the world press. In: Working Notes of the AAAI Spring Symposium in New Directions in Question Answering, Palo Alto, California (2003)

    Google Scholar 

  46. Wiebe, J., Wilson, T., Bruce, R., Bell, M., Martin, M.: Learning subjective language. Comput. Linguist. 30(3), 277–308 (2004)

    Article  Google Scholar 

  47. Wiebe, J., Wilson, T., Cardie, C.: Annotating expressions of opinions and emotions in language. Lang. Res. Eval. (formerly Computers and the Humanities) 39(2/3), 164–210 (2005)

    Google Scholar 

  48. Wiegand, M., Klakow, D.: Generalization methods for in-domain and cross-domain opinion holder extraction. In: Proceeding of the 13th Conference of the European Chapter of the Association for Computational Linguistics, (2012)

    Google Scholar 

  49. Wilks, Y., Bien, J.: Beliefs, points of view and multiple environments. Cogn. Sci. 7, 95–119 (1983)

    Article  Google Scholar 

  50. Wilson, T.: Annotating subjective content in meetings. In: Proceeding of the 6th Language Resources and Evaluations Conference, (2008)

    Google Scholar 

  51. Wilson, T.: Fine-grained subjectivity and sentiment analysis: recognizing the intensity, polarity, and attitudes of private states. Ph.D. Thesis, Intelligent Systems Program, University of Pittsburgh (2008)

    Google Scholar 

  52. Wilson, T., Wiebe, J., Hoffmann, P.: Recognizing contextual polarity: an exploration of features for phrase-level sentiment analysis. Comput. Linguist. 35(3), 399–433 (2009)

    Article  Google Scholar 

  53. Xiao, M., Guo, Y.: Multi-view AdaBoost for multilingual subjectivity analysis. In: Proceeding of the 24th International Conference on Computational Linguistics, (2012)

    Google Scholar 

  54. Yang, B., Cardie, C.: Joint inference for fine-grained opinion extraction. In: Proceeding of the 51st Annual Meeting of the Association for Computational Linguistics (vol. 1: Long Papers), (2013)

    Google Scholar 

  55. Yu, H., Hatzivassiloglou, V.: Towards answering opinion questions: separating facts from opinions and identifying the polarity of opinion sentences. In: Proceeding of the Conference on Empirical Methods in Natural Language Processing, (2003)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Science Foundation under grants IIS-0208798 and IIS-0208028, the Advanced Research and Development Activity (ARDA)’s Advanced Question Answering for Intelligence (AQUAINT) Program, and by the Northeast Regional Research Center (NRRC) which is sponsored by the Advanced Research and Development Activity (ARDA), a U.S. Government entity which sponsors and promotes research of import to the Intelligence Community which includes but is not limited to the CIA, DIA, NSA, NIMA, and NRO.

Author information

Authors and Affiliations

  1. Hanover College, 517 Ball Drive, Hanover, IN, 47243, USA

    Theresa Wilson

  2. University of Pittsburgh, 4200 Fifth Avenue, Pittsburgh, PA, 15260, USA

    Janyce Wiebe

  3. Cornell University, Ithaca, NY, 14850, USA

    Claire Cardie

Authors
  1. Theresa Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Janyce Wiebe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Claire Cardie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Janyce Wiebe .

Editor information

Editors and Affiliations

  1. Department of Computer Science, Vassar College, Poughkeepsie, New York, USA

    Nancy Ide

  2. Department of Computer Science, Volen Center for Complex Systems, Brandeis University, Waltham, Massachusetts, USA

    James Pustejovsky

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Wilson, T., Wiebe, J., Cardie, C. (2017). MPQA Opinion Corpus. In: Ide, N., Pustejovsky, J. (eds) Handbook of Linguistic Annotation. Springer, Dordrecht. https://doi.org/10.1007/978-94-024-0881-2_29

Download citation

  • DOI: https://doi.org/10.1007/978-94-024-0881-2_29

  • Published:

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-94-024-0879-9

  • Online ISBN: 978-94-024-0881-2

  • eBook Packages: Social SciencesSocial Sciences (R0)

Publish with us

Policies and ethics

Search

Navigation