Advertisement

Grading Chinese Answers on Specialty Subjective Questions

  • Dongjin Li
  • Tianyuan Liu
  • Wei Pan
  • Xiaoyue Liu
  • Yuqing SunEmail author
  • Feng Yuan
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1042)

Abstract

It is an important task to grade answers on specialty subjective questions, which is helpful for the supervision of human review and improving the efficiency and quality of review process. Since this grading process should be performed at the same time with human review, there are only a few samples available for each question that can be provided by specialty experts before review process. We investigate the problem of grading Chinese answers on specialty subjective questions with a reference answer in this paper by proposing a grading model that combines two Bi-LSTM networks with attention mechanism. The first part is a sequence to sequence Bi-LSTM network that adopts the pre-trained word embeddings as input. Since there is no embedding for some specialty words, we instead use the fine-grained word embeddings. After the max-pooling on each sentence, we adopt the mutual attention mechanism to learn the matching degree on specialty knowledge between each pair of sentences of answer and reference. Then we adopt another Bi-LSTM with max-pooling to have an overall vector. By concatenating these two vectors from answer and reference, a multilayer perceptron is adopted to predicate the scores. We adopt the real datasets on a national specialty examination to thoroughly verify the model performance against different amount of training data, network structures, pooling strategies and attention mechanisms. The experimental results show the effectiveness of our method.

Keywords

Grading Chinese answer Specialty subjective questions Attention mechanism 

Notes

Acknowledgments

This work was supported by the National Key R&D Program of China (Grant No. 2018YFC0831401), the National Natural Science Foundation of China (Grant No. 91646119), the Major Project of NSF Shandong Province (Grant No. ZR2018ZB0420), and the Key Research and Development Program of Shandong province (Grant No. 2017GGX10114). The scientific calculations in this paper have been done on the HPC Cloud Platform of Shandong University.

References

  1. 1.
    Burstein, J.: The E-rater® scoring engine: automated essay scoring with natural language processing. Shermis, M.D., Burstein, J.C. (eds.), pp. 113–121 (2003)Google Scholar
  2. 2.
    Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems 27, Annual Conference on Neural Information Processing Systems, pp. 3104–3112. MIT Press, Montreal (2014)Google Scholar
  3. 3.
    Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)zbMATHGoogle Scholar
  4. 4.
    Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1746–1751. ACL, Doha (2014)Google Scholar
  5. 5.
    Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. In: Advances in Neural Information Processing Systems 28, Annual Conference on Neural Information Processing Systems, pp. 649–657. ACL, Montreal (2015)Google Scholar
  6. 6.
    Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp. 655–665. ACL, Baltimore (2014)Google Scholar
  7. 7.
    Schwenk, H., Barrault, L., Conneau, A., LeCun, Y.: Very deep convolutional networks for text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, pp. 1107–1116. ACL, Valencia (2017)Google Scholar
  8. 8.
    Johnson, R., Zhang, T.: Deep pyramid convolutional neural networks for text categorization. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 562–570. ACL, Vancouver (2017)Google Scholar
  9. 9.
    Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1422–1432. ACL, Lisbon (2015)Google Scholar
  10. 10.
    Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent convolutional neural networks for text classification. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, pp. 2267–2273. AAAI, Austin (2015)Google Scholar
  11. 11.
    Shi, Y., Yao, K., Tian, L., Jiang, D.: Deep LSTM based feature mapping for query classification. In: The 2016 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 1501–1511. NAACL, San Diego (2016)Google Scholar
  12. 12.
    Xiao, Y., Cho, K.: Efficient character-level text classification by combining convolution and recurrent layers. arXiv:1602.00367 (2016)
  13. 13.
    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: 3rd International Conference on Learning Representations. San Diego (2015)Google Scholar
  14. 14.
    Tan, M., Xiang, B., Zhou, B.: LSTM-based deep learning models for non-factoid answer selection. arXiv:1511.04108 (2015)
  15. 15.
    Chaturvedi, A., Pandit, O.A., Garain, U.: CNN for text-based multiple choice question answering. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 272–277. ACL, Melbourne (2018)Google Scholar
  16. 16.
    Yang, Z., Yang, D., Dyer, C., He, X., Smola, A.J., Hovy, E.H.: Hierarchical attention networks for text classification. In: The 2016 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 1480–1489. NAACL, San Diego (2016)Google Scholar
  17. 17.
    Cui, Y., Liu, T., Chen, Z., Wang, S., Hu, G.: Consensus attention-based neural networks for Chinese reading comprehension. In: 26th International Conference on Computational Linguistics, pp. 1777–1786. ACM, Osaka (2016)Google Scholar
  18. 18.
    Cui, Y., Chen, Z., Wei, S., Wang, S., Liu, T., Hu, G.: Attention-over-attention neural networks for reading comprehension. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 593–602. ACL, Vancouver (2017)Google Scholar
  19. 19.
    Li, S., Zhao, Z., Hu, R., Li, W., Liu, T., Du, X.: Analogical reasoning on Chinese morphological and semantic relations. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 138–143. ACL, Melbourne (2018)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Dongjin Li
    • 1
    • 2
  • Tianyuan Liu
    • 1
    • 2
  • Wei Pan
    • 1
  • Xiaoyue Liu
    • 1
  • Yuqing Sun
    • 1
    Email author
  • Feng Yuan
    • 3
  1. 1.School of SoftwareShandong UniversityJinanChina
  2. 2.School of Computer Science and TechnologyShandong UniversityJinanChina
  3. 3.Shandong University Ouma Software Co., Ltd.JinanChina

Personalised recommendations