Cross-Lingual Semantic Textual Similarity Modeling Using Neural Networks

  • Xia LiEmail author
  • Minping Chen
  • Zihang Zeng
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 954)


Cross-lingual semantic textual similarity is to measure the semantic similarity of sentences in different languages. Previous work pay more attention on leveraging traditional NLP features (e.g., alignment features, syntactic features) to evaluate the semantic similarity of sentences. In this paper, we only use word embedding as basic features without any handcrafted features and build a model which is able to capture local and global semantic information of the sentences to evaluate semantic textual similarity. We test our model on SemEval-2017 and STS benchmark datasets. Our experiments show that our model improves the performance of the semantic textual similarity and achieves the best results compared with the baseline neural-network based methods reported on the two datasets.


Cross-lingual semantic textual similarity SemEval-2017 Neural networks 



This work is supported by the National Science Foundation of China (61402119) and Special Funds for the Cultivation of Guangdong College Students’ Scientific and Technological Innovation. (“Climbing Program” Special Funds.)


  1. 1.
    Agirre, E., et al.: Semeval-2016 Task 1: semantic textual similarity, monolingual and cross-lingual evaluation. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 497–511 (2016)Google Scholar
  2. 2.
    Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., Specia, L.: SemEval-2017 Task 1: semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055 (2017)
  3. 3.
    Tian, J., Zhou, Z., Lan, M., Wu, Y.: ECNU at SemEval-2017 Task 1: leverage kernel-based traditional nlp features and neural networks to build a universal model for multilingual and cross-lingual semantic textual similarity. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 191–197 (2017)Google Scholar
  4. 4.
    Wu, H., Huang, H.Y., Jian, P., et al.: BIT at SemEval-2017 Task 1: using semantic information space to evaluate semantic textual similarity. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 77–84 (2017)Google Scholar
  5. 5.
    Shao, Y.: HCTI at SemEval-2017 Task 1: use convolutional neural network to evaluate semantic textual similarity. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 130–133 (2017)Google Scholar
  6. 6.
    Cho, K., Van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014)
  7. 7.
    Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)
  8. 8.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014)Google Scholar
  9. 9.
    Wieting, J., Bansal, M., Gimpel, K., Livescu, K.: Towards universal paraphrastic sentence embeddings. arXiv preprint arXiv:1511.08198 (2015)
  10. 10.
    Agirre, E., Diab, M., Cer, D., Gonzalez-Agirre, A.: Semeval-2012 Task 6: a pilot on semantic textual similarity. In: Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the Main Conference and the Shared Task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation, pp. 385–393 (2012)Google Scholar
  11. 11.
    Agirre, E., Cer, D., Diab, M., Gonzalez-Agirre, A., Guo, W.: * SEM 2013 shared task: semantic textual similarity. In: Second Joint Conference on Lexical and Computational Semantics (* SEM), Volume 1: Proceedings of the Main Conference and the Shared Task: Semantic Textual Similarity, vol. 1, pp. 32–43 (2013)Google Scholar
  12. 12.
    Agirre, E., et al.: Semeval-2014 Task 10: multilingual semantic textual similarity. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pp. 81–91 (2014)Google Scholar
  13. 13.
    Agirre, E., et al.: Semeval-2015 Task 2: semantic textual similarity, English, Spanish and pilot on interpretability. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp. 252–263 (2015)Google Scholar
  14. 14.
    Hinton, G.E.: Learning distributed representations of concepts. In: Proceedings of the Eighth Annual Conference of the Cognitive Science Society, vol. 1, p. 12, (1986)Google Scholar
  15. 15.
    Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C.: A neural probabilistic language model. J. Mach. Learn. Res. 3(6), 1137–1155 (2003)zbMATHGoogle Scholar
  16. 16.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)Google Scholar
  17. 17.
    Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)Google Scholar
  18. 18.
    Kusner, M., Sun, Y., Kolkin, N., Weinberger, K.: From word embeddings to document distances. In: International Conference on Machine Learning, pp. 957–966 (2015)Google Scholar
  19. 19.
    Pagliardini, M., Gupta, P., Jaggi, M.: Unsupervised learning of sentence embeddings using compositional n-gram features. arXiv preprint arXiv:1703.02507 (2017)
  20. 20.
    Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075 (2015)
  21. 21.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Key Laboratory of Language Engineering and ComputingGuangdong University of Foreign StudiesGuangzhouChina
  2. 2.School of Information Science and Technology/School of Cyber SecurityGuangdong University of Foreign StudiesGuangzhouChina

Personalised recommendations