Abstract
Machine Translation of text is a fundamental problem in machine learning that resists solutions that do not take into account the dependencies between words and sentences. Recurrent Neural Networks have recently delivered outstanding results in learning about sequential dependencies in many languages. Arabic language as a target language has not received enough attention in the recent language model experiments due to its, structural and semantic difficulties. In this paper, we present a Statistical Machine Translation (SMT) Context Modelling using Recurrent Neural Networks (RNNs) and Latent Dirichlet Allocation (LDA). This research is based on the state-of-the-art RNN language model by Mikolov. Our preliminary contribution is in integrating and presenting a new hybridization to utilize Recurrent Neural Network sequential word learning dependencies as well as Latent Dirichlet Allocation context and topic classification ability to produce the most accurate language scoring.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Callison-Burch, C., Talbot, D., Osborne, M.: Statistical machine translation with word-and sentence-aligned parallel corpora. In: Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p. 175 (2004)
Durrani, N., Fraser, A., Schmid, H.: Model with minimal translation units, but decode with phrases. In: Proceedings of NAACL-HLT, 9–14 June 2013, Atlanta, Georgia (2013)
Brown, P., de Souza, P., Mercer, R., Pietra, V., Lai, J.: Class-based n-gram models of natural language. Computational Linguistics. Comput. Linguist. 18(4), 467–479 (1992)
Lipton, Z., Berkowitz, J., Elkan, C.: A Critical review of recurrent neural networks for sequence learning’, arXiv preprint arXiv:1506.00019 (2015)
Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1915–1929 (2013)
Zhao, B., Tam, Y.: Bilingual recurrent neural networks for improved statistical machine translation. In: Spoken Language Technology Workshop (SLT), 7–10 December 2014, South Lake Tahoe, NV. IEEE (2014)
Sundermeyer, M., Alkhouli, T., Wuebker, J., Ney, H.: Translation modeling with bidirectional recurrent neural networks. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 14–25 October 2014, Doha, Qatar (2014)
Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci. 79(8), 2554–2558 (1982)
Jordan, M.: Serial order: a parallel distributed processing approach. Technical report 8604, Institute for Cognitive Science, University of California, San Diego (1986)
Elman, J.: Finding structure in time. Cogn. Sci. 14, 179–211 (1990)
Mikolov, T., Zweig, G.: Context dependent recurrent neural network language model. In: 2012 workshop on Spoken Language Technology, pp. 234–239 (2012)
Kombrink, S., Mikolov, T., Karafiat, M., Burget, L.: Recurrent neural network based language model. In: INTERSPEECH 2010, 11th Annual Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, 26–30 September 2010, pp. 1045–1048 (2010)
Kalchbrenner, N., Blunsom, P.: Recurrent continuous translation models. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, October 2013, Seattle, Washington, USA, pp. 1700–1709 (2013)
Liu, S., Yang, N., Li, Zhou, M.: A recursive recurrent neural network for statistical machine translation. In: Proceedings of ACL, pp. 1491–1550 (2014)
Hu, Y., Auli, M., Gao, Q., Gao, J.: Minimum translation modeling with recurrent neural networks. In: Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, April 2014
Schuster, M., Paliwal, K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Schwenk, H.: Continuous space translation models for phrase-based statistical machine translation. In: 25th International Conference on Computational Linguistics (COLING), December, Mumbai, India, pp. 1071–1080 (2012)
Mikolov, T.: Statistical Language Models Based on Neural Networks. Ph.D., Brno University of Technology (2012)
Arabic Gigaword Corpus. https://catalog.ldc.upenn.edu/LDC2011T11
Mikolov, T.: RNNLM Toolkit (2012). http://www.rnnlm.org/. Accessed: 28 Nov 2015
Guessabi, F.: The cultural problems in translating a novel from arabic to english language. AWEJ Special Issue on Translation (2), 224–232 (2013)
https://ar.wikipedia.org/wiki/. Accessed: 20 Dec 2015
Ponweiser, M.: Latent dirichlet allocation in R. Diploma Thesis, Institute for Statistics and Mathematics 2 May 2012
Zhengxian, G., Guodong, Z.: Employing topic modeling for statistical machine translation. In: 2011 IEEE International Conference on Computer Science and Automation Engineering, CSAE (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Alsenan, S., Ykhlef, M. (2017). Statistical Machine Translation Context Modelling with Recurrent Neural Network and LDA. In: Hassanien, A., Shaalan, K., Gaber, T., Azar, A., Tolba, M. (eds) Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2016. AISI 2016. Advances in Intelligent Systems and Computing, vol 533. Springer, Cham. https://doi.org/10.1007/978-3-319-48308-5_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-48308-5_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-48307-8
Online ISBN: 978-3-319-48308-5
eBook Packages: EngineeringEngineering (R0)