Advertisement

Leveraging Arabic-English Bilingual Corpora with Crowd Sourcing-Based Annotation for Arabic-Hebrew SMT

  • Manish Gaurav
  • Guruprasad Saikumar
  • Amit Srivastava
  • Premkumar Natarajan
  • Shankar Ananthakrishnan
  • Spyros Matsoukas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7817)

Abstract

Recent studies in Statistical Machine Translation (SMT) paradigm have been focused on developing foreign language to English translation systems. However as SMT systems have matured, there is a lot of demand to translate from one foreign language to another language. Unfortunately, the availability of parallel training corpora for a pair of morphologically complex foreign languages like Arabic and Hebrew is very scarce. This paper uses active learning based data selection and crowd sourcing technique like Amazon Mechanical Turk to create Arabic-Hebrew parallel corpora. It then explores two different techniques to build Arabic-Hebrew SMT system. The first one involves the traditional cascading of two SMT systems using English as a pivot language. The second approach is training a direct Arabic-Hebrew SMT system using sentence pivoting. Finally, we use a phrase generalization approach to further improve our performance.

Keywords

Arabic-Hebrew Statistical Machine Translation Sentence Pivoting Amazon Mechanical Turk Active Learning Phrase Generalization 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Callison-Burch, C., Koehn, P., Os-borne, M.: Improved statistical ma-chine translation using paraphrases. In: Proceedings NAACL 2006 (2006)Google Scholar
  2. 2.
    Stefan, D., Munteanu, Marcu, D.: Improving machine translation perfor-mance by exploiting non-parallel corpora. Computational Linguistics 31(4), 477–504 (2005)CrossRefGoogle Scholar
  3. 3.
    Och, F.J., Ney, H.: A systematic comparison of various statistical alignment models. Computational Linguistics 29(1), 19–51 (2003)zbMATHCrossRefGoogle Scholar
  4. 4.
    Wu, H., Wang, H.: Revisiting Pivot Language Approach for Machine Translation. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, Suntec, Singapore, pp. 154–162. Association for Computational Linguistics (August 2009)Google Scholar
  5. 5.
    Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association of Computational Linguistics, ACL (2002)Google Scholar
  6. 6.
    Utiyama, M., Isahara, H.: Reliable measures for aligning Japanese-English news articles and sentences. In: ACL, pp. 72–79 (2003)Google Scholar
  7. 7.
    Utiyama, M., Isahara, H.: A Comparison of Pivot Methods for Phrase-Based Statistical Machine Translation. In: Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, Rochester, New York, pp. 484–491. Association for Computational Linguistics (April 2007)Google Scholar
  8. 8.
    Bertoldi, N., Barbaiani, M., Federi-co, M., Cattoni, R.: Phrase-Based Statistical Machine Translation with Pivot Languages. In: Proceedings of the International Workshop on Spoken Language Translation, Hawaii, USA, pp. 143–149 (2008)Google Scholar
  9. 9.
    Koehn, P., Och, F.J., Marcu, D.: Statistical phrase-based transla-tion. In: Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, Morristown, NJ, USA, pp. 48–54. Association for Computational Linguistics (2003)Google Scholar
  10. 10.
    Resnik, P., Smith, N.A.: The web as a parallel corpus. Computational Linguistics 29(3), 349–380 (2003)CrossRefGoogle Scholar
  11. 11.
    Zbib, R., Malchiodi, E., Devlin, J., Stallard, D., Matsoukas, S., Schwartz, R., Makhoul, J., Zaidany, O.F., Callison-Burch, C.: Machine Translation of Arabic Dialects. In: Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2012)Google Scholar
  12. 12.
    Ananthakrishnan, S., Vitaladevuni, S., Prasad, R., Natarajan, P.: Source Error-Projection for Sample Selection in Phrase-Based SMT for Resource-Poor Languages. In: Proceedings of the IJCNLP 2011 (2011)Google Scholar
  13. 13.
    Cohn, T., Lapata, M.: Machine Translation by Triangulation: Making Ef-fective Use of Multi-Parallel Corpora. In: Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, Prague, Czech Republic, pp. 728–735 (2007)Google Scholar
  14. 14.
    Ambati, V., Vogel, S.: Can crowds build parallel corpora for machine translation systems? In: NAACL Workshop on Creating Speech and Language Data With Amazon’s Mechanical Turk (2010)Google Scholar
  15. 15.
    Ambati, V., Vogel, S., Carbonell, J.: Active learning and crowd-sourcing for machine translation. In: Proceedings of Language Resources and Evaluation, LREC (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Manish Gaurav
    • 1
  • Guruprasad Saikumar
    • 1
  • Amit Srivastava
    • 1
  • Premkumar Natarajan
    • 1
  • Shankar Ananthakrishnan
    • 1
  • Spyros Matsoukas
    • 1
  1. 1.Raytheon BBN TechnologiesCambridgeUSA

Personalised recommendations