Can Text Summaries Help Predict Ratings? A Case Study of Movie Reviews

  • Horacio Saggion
  • Elena Lloret
  • Manuel Palomar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7337)


This paper presents an analysis of the rating inference task – the task of correctly predicting the rating associated to a review, in the context of movie reviews. For achieving this objective, we study the use of automatic text summaries instead of the full reviews. An extrinsic evaluation framework is proposed, where full reviews and different types of summaries (positional, generic and sentiment-based) of several compression rates (from 10% to 50%) are evaluated. We are facing a difficult task; however, the results obtained are very promising and demonstrate that summaries are appropriate for the rating inference problem, performing at least equally to the full reviews when summaries are at least 30% compression rate. Moreover, we also find out that the way the review is organised, as well as the style of writing, strongly determines the performance of the different types of summaries.


Content representation and processing Natural Language Processing Text Summarization Rating Inference Opinion Classification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cunningham, H., Maynard, D., Bontcheva, K., Tablan, V.: GATE: A Framework and Graphical Development Environment for Robust NLP Tools and Applications. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, USA (2002)Google Scholar
  2. 2.
    Goldberg, A.B., Zhu, X.: Seeing stars when there aren’t many stars: graph-based semi-supervised learning for sentiment categorization. In: Proceedings of the 1st Workshop on Graph Based Methods for NLP, pp. 45–52 (2006)Google Scholar
  3. 3.
    Li, Y., Bontcheva, K., Cunningham, H.: Adapting SVM for Data Sparseness and Imbalance: A Case Study in Information Extraction. Natural Language Engineering 15(2), 241–271 (2009)CrossRefGoogle Scholar
  4. 4.
    Lloret, E.: Text Summarisation based on Human Language Technologies and its Applications. Ph.D. thesis, University of Alicante (2011)Google Scholar
  5. 5.
    Lloret, E., Saggion, H., Palomar, M.: Experiments on Summary-based Opinion Classification. In: Proceedings of the Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pp. 107–115 (2010)Google Scholar
  6. 6.
    Pang, B., Lee, L.: Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales. In: Proceedings of the Association of Computational Linguistics, pp. 115–124 (2005)Google Scholar
  7. 7.
    Pang, B., Lee, L.: Opinion Mining and Sentiment Analysis. Foundations and Trends in Information Retrieval 2(1-2), 1–135 (2008)CrossRefGoogle Scholar
  8. 8.
    Qu, L., Ifrim, G., Weikum, G.: The bag-of-opinions method for review rating prediction from sparse text patterns. In: Proceedings of the 23rd International Conference on Computational Linguistics, pp. 913–921 (2010)Google Scholar
  9. 9.
    Saggion, H., Funk, A.: Extracting Opinions and Facts for Business Intelligence. RNTI E-17, 119–146 (2009)Google Scholar
  10. 10.
    Saggion, H., Lloret, E., Palomar, M.: Using Text Summaries for Predicting Rating Scales. In: Proceedings of the 1st Workshop on Computational Approaches to Subjectivity and Sentiment Analysis (2010)Google Scholar
  11. 11.
    Spärck Jones, K.: Automatic Summarising: The State of the Art. Information Processing & Management 43(6), 1449–1481 (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Horacio Saggion
    • 1
  • Elena Lloret
    • 2
  • Manuel Palomar
    • 2
  1. 1.Department of Information and Communication TechnologiesUniversitat Pompeu FabraBarcelonaSpain
  2. 2.Department of Software and Computing SystemsUniversity of AlicanteSpain

Personalised recommendations