Advertisement

Multi-Perspective Fusion Network for Commonsense Reading Comprehension

  • Chunhua Liu
  • Yan Zhao
  • Qingyi Si
  • Haiou Zhang
  • Bohan Li
  • Dong Yu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11221)

Abstract

Commonsense Reading Comprehension (CRC) is a significantly challenging task, aiming at choosing the right answer for the question referring to a narrative passage, which may require commonsense knowledge inference. Most of the existing approaches only fuse the interaction information of choice, passage, and question in a simple combination manner from a union perspective, which lacks the comparison information on a deeper level. Instead, we propose a Multi-Perspective Fusion Network (MPFN), extending the single fusion method with multiple perspectives by introducing the difference and similarity fusion. More comprehensive and accurate information can be captured through the three types of fusion. We design several groups of experiments on MCScript dataset [11] to evaluate the effectiveness of the three types of fusion respectively. From the experimental results, we can conclude that the difference fusion is comparable with union fusion, and the similarity fusion needs to be activated by the union fusion. The experimental result also shows that our MPFN model achieves the state-of-the-art with an accuracy of 83.52% on the official test set.

Keywords

Commonsense Reading Comprehension Fusion Network Multi-perspective 

Notes

Acknowledgements

This work is funded by Beijing Advanced Innovation for Language Resources of BLCU, the Fundamental Research Funds for the Central Universities in BLCU (17PT05), the Natural Science Foundation of China (61300081), and the Graduate Innovation Fund of BLCU (No. 18YCX010).

References

  1. 1.
    Chen, Q., Zhu, X., Ling, Z.H., Wei, S., Jiang, H., Inkpen, D.: Enhanced LSTM for natural language inference. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1657–1668. Association for Computational Linguistics, Vancouver, July 2017. http://aclweb.org/anthology/P17-1152
  2. 2.
    Chen, Z., Cui, Y., Ma, W., Wang, S., Liu, T., Hu, G.: HFL-RC system at SemEval-2018 task 11: hybrid multi-aspects model for commonsense reading comprehension. arXiv preprint arXiv:1803.05655 (2018)
  3. 3.
    Merkhofer, E., Henderson, J., Bloom, D., Strickhart, L., Zarrella, G.: MITRE at SemEval-2018 task 11: commonsense reasoning without commonsense knowledge (2018)Google Scholar
  4. 4.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997).  https://doi.org/10.1162/neco.1997.9.8.1735CrossRefGoogle Scholar
  5. 5.
    Huang, H., Zhu, C., Shen, Y., Chen, W.: FusionNet: fusing via fully-aware attention with application to machine comprehension. CoRR abs/1711.07341 (2017)Google Scholar
  6. 6.
    Joshi, M., Choi, E., Weld, D.S., Zettlemoyer, L.: TriviaQA: a large scale distantly supervised challenge dataset for reading comprehension. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Vancouver, July 2017Google Scholar
  7. 7.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014)Google Scholar
  8. 8.
    Lai, G., Xie, Q., Liu, H., Yang, Y., Hovy, E.H.: RACE: large-scale reading comprehension dataset from examinations. In: EMNLP (2017)Google Scholar
  9. 9.
    Lee, K., Kwiatkowski, T., Parikh, A.P., Das, D.: Learning recurrent span representations for extractive question answering. CoRR abs/1611.01436 (2016). http://arxiv.org/abs/1611.01436
  10. 10.
    Mou, L., et al.: Natural language inference by tree-based convolution and heuristic matching. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 130–136. Association for Computational Linguistics, Berlin, August 2016. http://anthology.aclweb.org/P16-2022
  11. 11.
    Ostermann, S., Modi, A., Roth, M., Thater, S., Pinkal, M.: MCScript: a novel dataset for assessing machine comprehension using script knowledge. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), 7–12 May 2018, Miyazaki, Japan. European Language Resources Association (ELRA) (2018)Google Scholar
  12. 12.
    Parikh, A., Täckström, O., Das, D., Uszkoreit, J.: A decomposable attention model for natural language inference. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2249–2255. Association for Computational Linguistics (2016).  https://doi.org/10.18653/v1/D16-1244, http://www.aclweb.org/anthology/D16-1244
  13. 13.
    Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: EMNLP (2014)Google Scholar
  14. 14.
    Rajpurkar, P., Zhang, J., Lopyrev, K., Liang, P.: Squad: 100, 000+ questions for machine comprehension of text. CoRR abs/1606.05250 (2016)Google Scholar
  15. 15.
    Richardson, M., Burges, C.J.C., Renshaw, E.: MCTest: a challenge dataset for the open-domain machine comprehension of text. In: EMNLP (2013)Google Scholar
  16. 16.
    Seo, M.J., Kembhavi, A., Farhadi, A., Hajishirzi, H.: Bidirectional attention flow for machine comprehension. CoRR abs/1611.01603 (2016)Google Scholar
  17. 17.
    Trischler, A., et al.: NewsQA: a machine comprehension dataset. In: Rep4NLP@ACL (2017)Google Scholar
  18. 18.
    Wang, L., Sun, M., Zhao, W., Shen, K., Liu, J.: Yuanfudao at SemEval-2018 task 11: three-way attention and relational knowledge for commonsense machine comprehension. In: SemEval@NAACL-HLT, pp. 758–762. Association for Computational Linguistics (2018)Google Scholar
  19. 19.
    Wang, S., Jiang, J.: Learning natural language inference with LSTM. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1442–1451. Association for Computational Linguistics, San Diego, June 2016. http://www.aclweb.org/anthology/N16-1170
  20. 20.
    Wang, W., Yang, N., Wei, F., Chang, B., Zhou, M.: Gated self-matching networks for reading comprehension and question answering. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 189–198. Association for Computational Linguistics (2017).  https://doi.org/10.18653/v1/P17-1018, http://www.aclweb.org/anthology/P17-1018
  21. 21.
    Weissenborn, D., Wiese, G., Seiffe, L.: FastQA: a simple and efficient neural architecture for question answering. CoRR abs/1703.04816 (2017). http://arxiv.org/abs/1703.04816
  22. 22.
    Weston, J., Bordes, A., Chopra, S., Mikolov, T.: Towards AI-complete question answering: a set of prerequisite toy tasks. CoRR abs/1502.05698 (2015)Google Scholar
  23. 23.
    Xia, J.: Jiangnan at SemEval-2018 task 11: deep neural network with attention method for machine comprehension task (2018)Google Scholar
  24. 24.
    Xiong, C., Zhong, V., Socher, R.: DCN+: mixed objective and deep residual coattention for question answering. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=H1meywxRW
  25. 25.
    Xu, Y., Liu, J., Gao, J., Shen, Y., Liu, X.: Towards human-level machine reading comprehension: reasoning and inference with multiple strategies. CoRR abs/1711.04964 (2017)Google Scholar
  26. 26.
    Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: NAACL, pp. 1480–1489. Association for Computational Linguistics, San Diego, June 2016. http://www.aclweb.org/anthology/N16-1174
  27. 27.
    Zhu, H., Wei, F., Qin, B., Liu, T.: Hierarchical attention flow for multiple-choice reading comprehension. In: AAAI (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Chunhua Liu
    • 1
  • Yan Zhao
    • 1
  • Qingyi Si
    • 1
  • Haiou Zhang
    • 1
  • Bohan Li
    • 1
  • Dong Yu
    • 1
    • 2
  1. 1.Beijing Language and Culture UniversityBeijingChina
  2. 2.Beijing Advanced Innovation for Language Resources of BLCUBeijingChina

Personalised recommendations