Abstract
Neural machine translation (NMT) has shown promising progress in recent years. However, for reducing the computational complexity, NMT typically needs to limit its vocabulary scale to a fixed or relatively acceptable size, which leads to the problem of rare word and out-of-vocabulary (OOV). In this paper, we present that the semantic concept information of word can help NMT learn better semantic representation of word and improve the translation accuracy. The key idea is to utilize the external semantic knowledge base WordNet to replace rare words and OOVs with their semantic concepts of WordNet synsets. More specifically, we propose two semantic similarity models to obtain the most similar concepts of rare words and OOVs. Experimental results on 4 translation tasks (We verify the effectiveness of our method on four translation tasks, including English-to- German, German-to-English, English-to-Chinese and Chinese-to-English.) show that our method outperforms the baseline RNNSearch by 2.38–2.88 BLEU points. Furthermore, the proposed hybrid method by combining BPE and our proposed method can also gain 0.39–0.97 BLEU points improvement over BPE. Experiments and analysis presented in this study also demonstrate that the proposed method can significantly improve translation quality of OOVs in NMT.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR 2015 (2015)
Vaswani, A., et al.: Attention is all you need. CoRR abs/1706.03762 (2017)
Wu, Y., Schuster, M., Chen, Z., et al.: Google’s neural machine translation system: bridging the gap between human and machine translation (2016)
Luong, M.T., Sutskever, I., Le, Q.V., et al.: Addressing the rare word problem in neural machine translation. Bull. Univ. Agric. Sci. Vet. Med. Cluj-Napoca. Vet. Med. 27(2), 82–86 (2014)
Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. Computer Science (2015)
Miller, G.A.: WordNet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)
Palangi, H., Palangi, H., Deng, L., Shen, Y.: Deep sentence embedding using long short-term memory networks: analysis and application to information retrieval. IEEE/ACM Trans. Audio Speech Lang. Process. 24(4), 694–707 (2015)
Papineni, K., Roukos, S., et al.: BLEU: a method for automatic valuation of machine translation. In: Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, July 2002, pp. 311–318 (2002)
Ziemski, M., Junczys-Dowmunt, M., Pouliquen, B.: The United Nations Parallel Corpus, Language Resources and Evaluation (LREC 2016), Portorož, Slovenia, May 2016 (2016)
Stolcke, A.: SRILM—an extensible language modeling toolkit. In: International Conference on Spoken Language Processing, pp. 901–904 (2002)
Li, X., Zhang, J., Zong, C.: Towards zero unknown word in neural machine translation. In: International Joint Conference on Artificial Intelligence, pp. 2852–2858. AAAI Press (2016)
Li, S., Xu, J., Miao, G., Zhang, Y., Chen, Y.: A semantic concept based unknown words processing method in neural machine translation. In: Huang, X., Jiang, J., Zhao, D., Feng, Y., Hong, Yu. (eds.) NLPCC 2017. LNCS (LNAI), vol. 10619, pp. 233–242. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-73618-1_20
Sutskever, I., et al.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)
Acknowledgments
The research work has been supported by the National Nature Science Foundation of China (Contract 61370130, 61473294 and 61502149), and Beijing Natural Science Foundation under Grant No. 4172047, and the Fundamental Research Funds for the Central Universities (2015JBM033), and the International Science and Technology Cooperation Program of China under grant No. 2014DFA11350.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, F., Xu, J., Miao, G., Chen, Y., Zhang, Y. (2019). Improving Performance of NMT Using Semantic Concept of WordNet Synset. In: Chen, J., Zhang, J. (eds) Machine Translation. CWMT 2018. Communications in Computer and Information Science, vol 954. Springer, Singapore. https://doi.org/10.1007/978-981-13-3083-4_4
Download citation
DOI: https://doi.org/10.1007/978-981-13-3083-4_4
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-3082-7
Online ISBN: 978-981-13-3083-4
eBook Packages: Computer ScienceComputer Science (R0)