Abstract
An inherent weakness of neural machine translation (NMT) systems is their inability to correctly translate unknown words. Traditional unknown words processing methods are usually based on word vectors trained on large scale of monolingual corpus. Replacing the unknown words according to the similarity of word vectors. However, it suffers from two weaknesses: Firstly, the resulting vectors of unknown words are not of high quality; Secondly, it is difficult to deal with polysemous words. This paper proposes an unknown word processing method by integrating HowNet. Using the concepts and sememes in HowNet to seek the replacement words of unknown words. Experimental results show that our proposed method can not only improves the performance of NMT, but also provides some advantages compared with the traditional unknown words processing methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kalchbrenner, N., Blunsom, P.: Recurrent continuous translation models (2013)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. Adv. Neural. Inf. Process. Syst. 4, 3104–3112 (2014)
Luong, M.T., Pham, H., Manning, C.D.: On the properties of neural machine translation: encoder-decoder approaches. Computer Science (2014)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Computer Science (2014)
Li, X., Zhang, J., Zong, C.: Towards zero unknown word in neural machine translation. In: International Joint Conference on Artificial Intelligence, pp. 2852–2858. AAAI Press (2016)
Liu, Q., Li, S.: Word similarity computing based on Hownet. Comput. Linguist. Chin. Lang. Process. 7(2), 59–76 (2002)
Och, F.J., Ney, H.: A Systematic Comparison of Various Statistical Alignment Models. MIT Press, Cambridge (2003)
Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: BLEU: a method for auto matice valuation of machine translation. In: Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, July 2002, pp. 311–318 (2002)
Zeiler, M.D.: ADADELTA: an adaptive learning rate method. Computer Science (2012)
Srivastava, N., Hinton, G., Krizhevsky, A., et al.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Collins, M., Koehn, P.: Clause restructuring for statistical machine translation. In: Meeting on Association for Computational Linguistics. Association for Computational Linguistics, pp. 531–540 (2005)
Meng, F., Lu, Z., Li, H., et al.: Interactive attention for neural machine translation (2016)
Luong, M.T., Sutskever, I., Le, Q.V., et al.: Addressing the rare word problem in neural machine translation. Bull. Univ. Agric. Sci. Vet. Med. Cluj-Napoca 27(2), 82–86 (2014)
Mikolov, T., Sutskever, I., Chen, K., et al.: Distributed representations of words and phrases and their compositionality. Adv. Neural. Inf. Process. Syst. 26, 3111–3119 (2013)
Stolcke, A.: SRILM—an extensible language modeling toolkit. In: International Conference on Spoken Language Processing, pp. 901–904 (2002)
Acknowledgments
The authors are supported by the National Nature Science Foundation of China (Contract 61370130 and 61473294), and Beijing Natural Science Foundation under Grant No. 4172047, and the Fundamental Research Funds for the Central Universities (2015JBM033), and the International Science and Technology Cooperation Program of China under grant No. 2014DFA11350.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, S., Xu, J., Zhang, Y., Chen, Y. (2017). A Method of Unknown Words Processing for Neural Machine Translation Using HowNet. In: Wong, D., Xiong, D. (eds) Machine Translation. CWMT 2017. Communications in Computer and Information Science, vol 787. Springer, Singapore. https://doi.org/10.1007/978-981-10-7134-8_3
Download citation
DOI: https://doi.org/10.1007/978-981-10-7134-8_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-7133-1
Online ISBN: 978-981-10-7134-8
eBook Packages: Computer ScienceComputer Science (R0)