Can Text Summaries Help Predict Ratings? A Case Study of Movie Reviews
This paper presents an analysis of the rating inference task – the task of correctly predicting the rating associated to a review, in the context of movie reviews. For achieving this objective, we study the use of automatic text summaries instead of the full reviews. An extrinsic evaluation framework is proposed, where full reviews and different types of summaries (positional, generic and sentiment-based) of several compression rates (from 10% to 50%) are evaluated. We are facing a difficult task; however, the results obtained are very promising and demonstrate that summaries are appropriate for the rating inference problem, performing at least equally to the full reviews when summaries are at least 30% compression rate. Moreover, we also find out that the way the review is organised, as well as the style of writing, strongly determines the performance of the different types of summaries.
KeywordsContent representation and processing Natural Language Processing Text Summarization Rating Inference Opinion Classification
Unable to display preview. Download preview PDF.
- 1.Cunningham, H., Maynard, D., Bontcheva, K., Tablan, V.: GATE: A Framework and Graphical Development Environment for Robust NLP Tools and Applications. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, USA (2002)Google Scholar
- 2.Goldberg, A.B., Zhu, X.: Seeing stars when there aren’t many stars: graph-based semi-supervised learning for sentiment categorization. In: Proceedings of the 1st Workshop on Graph Based Methods for NLP, pp. 45–52 (2006)Google Scholar
- 4.Lloret, E.: Text Summarisation based on Human Language Technologies and its Applications. Ph.D. thesis, University of Alicante (2011)Google Scholar
- 5.Lloret, E., Saggion, H., Palomar, M.: Experiments on Summary-based Opinion Classification. In: Proceedings of the Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pp. 107–115 (2010)Google Scholar
- 6.Pang, B., Lee, L.: Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales. In: Proceedings of the Association of Computational Linguistics, pp. 115–124 (2005)Google Scholar
- 8.Qu, L., Ifrim, G., Weikum, G.: The bag-of-opinions method for review rating prediction from sparse text patterns. In: Proceedings of the 23rd International Conference on Computational Linguistics, pp. 913–921 (2010)Google Scholar
- 9.Saggion, H., Funk, A.: Extracting Opinions and Facts for Business Intelligence. RNTI E-17, 119–146 (2009)Google Scholar
- 10.Saggion, H., Lloret, E., Palomar, M.: Using Text Summaries for Predicting Rating Scales. In: Proceedings of the 1st Workshop on Computational Approaches to Subjectivity and Sentiment Analysis (2010)Google Scholar