Machine learning using context vectors for object coreference resolution

Computing (2021)Cite this article

Abstract

Object coreference resolution is used in sentiment analysis to identify sentiment words referring to an aspect of an object in a document. However, this poses a challenge in natural language processing and is consequently an area of ongoing research. Further, to the best of our knowledge, object coreference resolution with more than one object has not been given much attention. To effectively address object coreference resolution, this paper proposes a method in which machine learning is applied to a large volume of textual data represented by context vectors, constituting a new form of language representation. The proposed machine learning model uses these vectors to achieve state-of-the-art performance in object coreference resolution. In addition, a combination of dependency grammar, sentiment ontology, and coreference graphs is used to obtain triplets of object, aspect, and sentiment. In experiments conducted on sentiment textual data obtained from Amazon.com, the proposed method achieved an average coreference resolution of object, aspect, and sentiment precision value of approximately 90%. This result suggests that the proposed method can contribute considerably to the field of object coreference resolution, and further research is therefore warranted.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Notes

  1. 1.

    https://www.amazon.com/ and http://snap.stanford.edu/data/.

  2. 2.

    http://www.younetmedia.com.

References

  1. 1.

    Witte R, Bergler S (2003) Fuzzy coreference resolution for summarization. In: Proceedings of 2003 international symposium on reference resolution and its applications to question answering and summarization (ARQAS)

  2. 2.

    Hartrumpf S, Glöckner I, Leveling J (2008) Coreference resolution for questions and answer merging by validation. In: Peters C et al (eds) Advances in multilingual and multimodal information retrieval. CLEF 2007, pp 269–272

  3. 3.

    Le Thi T, Thanh TQ, Thi TP (2017) Ontology-based entity coreference resolution for sentiment analysis. In: Proceedings of the eighth international symposium on information and communication technology (SoICT), ACM, New York, NY, USA, pp 50–56

  4. 4.

    Liu B (2012) Sentiment analysis and opinion mining. Morgan & Claypool Publishers, San Rafael

    Google Scholar 

  5. 5.

    Sukthanker R, Poria S, Cambria E, Thirunavukarasu R (2018) Anaphora and coreference resolution: a review. arXiv:1805.11824

  6. 6.

    Sukthanker R, Poria S, Cambria E, Thirunavukarasu R (2020) Anaphora and coreference resolution: a review. Inf Fusion 59:139–162

    Article  Google Scholar 

  7. 7.

    Lu J, Ng V (2018) Event coreference resolution: a survey of two decades of research. In: Proceedings of the twenty-seventh international joint conference on artificial intelligence survey track, pp 5479–5486.https://doi.org/10.24963/ijcai.2018/773

  8. 8.

    Ng V (2017) Machine learning for entity coreference resolution: a retrospective look at two decades of research. In: Proceedings of the thirty-first AAAI conference on artificial intelligence, AAAI Press, pp 4877–4884

  9. 9.

    Hobbs JR (1978) Resolving pronoun references. Lingua 44(4):311–338

    Article  Google Scholar 

  10. 10.

    Lappin S, Leass HJ (1994) An algorithm for pronominal anaphora resolution. Comput Linguist 20(4):535–561

    Google Scholar 

  11. 11.

    Cardie C, Wagstaff K (1999) Noun phrase coreference as clustering. In: Empirical methods in natural language processing conference (EMNLP 1999), pp 82–89

  12. 12.

    Ng V, Cardie C (2002) Improving machine learning approaches to coreference resolution. In: Proceedings of the 40th annual meeting on association for computational linguistics, Association for Computational Linguistics, pp 104–111

  13. 13.

    Pandian A, Mulaffer L, Oflazer K, AlZeyara A (2020) Precision event coreference resolution using neural network classifiers. Computación y Sistemas. https://doi.org/10.13053/cys-1-1-3349

    Article  Google Scholar 

  14. 14.

    Wiseman S, Rush AM, Shieber S, Weston J (2015) Learning anaphoricity and antecedent ranking features for coreference resolution. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: long papers), vol 1, pp 1416–1426

  15. 15.

    Cheng X, Voigt R (2015) A deep architecture for coreference resolution. In: Proceedings of the 2001 workshop on computational natural language, pp 1–8

  16. 16.

    Wiseman S, Rush AM, Shieber SM (2016) Learning global features for coreference resolution. arXiv:1604.03035

  17. 17.

    Clark K, Manning CD (2016) Deep reinforcement learning for mention-ranking coreference models. arXiv:1609.08667

  18. 18.

    Clark K, Manning CD (2016) Improving coreference resolution by learning entity-level distributed representations. arXiv:1606.0132

  19. 19.

    Nguyen TH, Meyers A, Grishman R (2016) New York University 2016 system for KBP event nugget: a deep learning approach. In: Proceedings of the text analysis conference

  20. 20.

    Lee K, He L, Lewis M, Zettlemoyer L (2017) End-to-end neural coreference resolution. arXiv:1707.07045

  21. 21.

    Ding X, Liu B (2010) Resolving object and attribute coreference in opinion mining. In: Proceedings of the 23rd international conference on computational linguistics (COLING-2010), pp 268–276

  22. 22.

    Zhao Y, Qin B, Liu T (2014) Aspect-object alignment using integer linear programming. In: Zong C, Nie JY, Zhao D, Feng Y (eds) Natural language processing and Chinese computing, vol 496. Communications in computer and information science. Springer, Berlin, pp 193–204

    Google Scholar 

  23. 23.

    Yadav A, Vishwakarma DK (2020) Sentiment analysis using deep learning architectures: a review. Artif Intell Rev 53:4335–4385. https://doi.org/10.1007/s10462-019-09794-5

    Article  Google Scholar 

  24. 24.

    Kharde VA, Sonawane S (2016) Sentiment analysis of Twitter data: a survey of techniques. Int J Comput Appl 139(11):5–15

    Google Scholar 

  25. 25.

    Appel O, Chiclana F, Carter J, Fujita H (2016) A hybrid approach to the sentiment analysis problem at the sentence level. Knowl Based Syst 108:110–124

    Article  Google Scholar 

  26. 26.

    Fang X, Zhan J (2015) Sentiment analysis using product review data. J Big Data 2(1):5

    Article  Google Scholar 

  27. 27.

    Fu X, Liu W, Xu Y, Yu C, Wang T (2016) Long short-term memory network over rhetorical structure theory for sentence-level sentiment analysis. In: Asian conference on machine learning, pp 17–32

  28. 28.

    Akhtar MS, Gupta D, Ekbal A, Bhattacharyya P (2017) Feature selection and ensemble construction: a two-step method for aspect-based sentiment analysis. Knowl Based Syst 125:116–135

    Article  Google Scholar 

  29. 29.

    Wu C, Wu F, Wu S, Yuan Z, Liu J, Huang H (2019) Semi-supervised dimensional sentiment analysis with variational autoencoder. Knowl Based Syst 165:30–39

    Article  Google Scholar 

  30. 30.

    Kumar S, Gahalawat M, Roy PP, Dogra DP, Kim B-GJE (2020) Exploring impact of age and gender on sentiment analysis using machine learning. Electronics 9:374

    Article  Google Scholar 

  31. 31.

    Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: Proceedings of the international conference on machine learning (ICML 2014)

  32. 32.

    Zhou X, Wan X, Xiao J (2016) Attention-based LSTM network for cross-lingual sentiment classification. In: Proceedings of the conference on empirical methods in natural language processing (EMNLP 2016)

  33. 33.

    Yin Y, Song Y, Zhang M (2017) Document-level multi-aspect sentiment classification as machine comprehension. In: Proceedings of the conference on empirical methods in natural language processing (EMNLP 2017)

  34. 34.

    Tang D, Zhang M (2018) Deep learning in sentiment analysis. In: Deep Learning in natural language processing. Springer, Berlin, pp 219–253

  35. 35.

    Schmitt M, Steinheber S, Schreiber K, Roth B (2018) Joint aspect and polarity classification for aspect-based sentiment analysis with end-to-end neural networks. arXiv:1808.09238

  36. 36.

    Do HH, Prasad P, Maag A, Alsadoon AJ (2019) Deep learning for aspect-based sentiment analysis: a comparative review. Expert Syst Appl 118:272–299

    Article  Google Scholar 

  37. 37.

    Soni S, Sharaff A (2015) Sentiment analysis of customer reviews based on hidden Markov model. In: Proceedings of the 2015 international conference on advanced research in computer science engineering & technology (ICARCSET 2015), Unnao, India, 6 March 2015, pp 1–5

  38. 38.

    Salas-Zàrate MP, Medina-Moreira J, Lagos-Ortiz K, Luna-Aveiga H, Rodriguez-Garcia MA, Valencia-García RJC (2017) Sentiment analysis on tweets about diabetes: an aspect-level approach. Comput Math Methods Med 2017:5140631

    Article  Google Scholar 

  39. 39.

    Akhtar MS, Kumar A, Ekbal A, Bhattacharyya P (2016) A hybrid deep learning architecture for sentiment analysis. In: Proceedings of the international conference on computational linguistics (COLING 2016)

  40. 40.

    Pandey AC, Rajpoot DS, Saraswat M (2017) Twitter sentiment analysis using hybrid cuckoo search method. Inf Process Manag 53:764–779

    Article  Google Scholar 

  41. 41.

    Widrow B, Lehr MA (1990) 30 years of adaptive neural networks: perceptron, madaline, and backpropagation. Proc IEEE 78(9):1415–1442

    Article  Google Scholar 

  42. 42.

    Olazaran M (1996) A sociological study of the official history of the perceptrons controversy. Soc Stud Sci 26(3):611–659

    Article  Google Scholar 

  43. 43.

    Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv:1301.3781

  44. 44.

    Pennington J, Socher R, Manning C (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  45. 45.

    Devlin J, Chang MW, Lee K, Toutanova K (2018) BERT: pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805

  46. 46.

    Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30:6000–6010

    Google Scholar 

  47. 47.

    Golan JS (1995) Foundations of linear algebra. Kluwer Academic Publishers, Springer

    Google Scholar 

Download references

Funding

This research is funded by Ho Chi Minh City University of Technology (HCMUT), VNU-HCM, under Grant Number BK-SDH-2021-1585007.

Author information

Affiliations

  1. Faculty of Computer Science & Engineering, Ho Chi Minh City University of Technology (HCMUT), Ho Chi Minh City, Vietnam

    Thuy Le Thi, Tuoi Phan Thi & Tho Quan Thanh

  2. Vietnam National University, Ho Chi Minh City, Vietnam

    Thuy Le Thi, Tuoi Phan Thi & Tho Quan Thanh

  3. Faculty of Information Technology, Industrial University of Ho Chi Minh City, Ho Chi Minh City, Vietnam

    Thuy Le Thi

Authors
  1. Thuy Le Thi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tuoi Phan Thi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tho Quan Thanh
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Writing-original draft: TLT; writing-review & editing: TPT, TQT.

Corresponding author

Correspondence to Thuy Le Thi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Le Thi, T., Phan Thi, T. & Quan Thanh, T. Machine learning using context vectors for object coreference resolution. Computing (2021). https://doi.org/10.1007/s00607-021-00902-4

Download citation

Keywords

  • Object coreference resolution
  • Object aspect
  • Sentiment analysis
  • Sentiment ontology
  • CROAS

Mathematics Subject Classification

  • 68T50