Advertisement

Data Selection via Semi-supervised Recursive Autoencoders for SMT Domain Adaptation

  • Yi Lu
  • Derek F. Wong
  • Lidia S. Chao
  • Longyue Wang
Part of the Communications in Computer and Information Science book series (CCIS, volume 493)

Abstract

In this paper, we present a novel data selection approach based on semi-supervised recursive autoencoders. The model is trained to capture the domain specific features and used for detecting sentences, which are relevant to a specific domain, from a large general-domain corpus. The selected data are used for adapting the built language model and translation model to target domain. Experiments were conducted on an in-domain (IWSLT2014 Chinese-English TED Talk) and a general-domain corpus (UM-Corpus). We evaluated the proposed data selection model in both intrinsic and extrinsic evaluations to investigate the selection successful rate (F-score) of pseudo data, as well as the translation quality (BLEU score) of adapting SMT systems. Empirical results reveal the proposed approach outperforms the state-of-the-art selection approach.

Keywords

Statistical Machine Translation Domain Adaptation Data Selection Semi-Supervise Recursive Autoencoders 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Axelrod, A., He, X., Gao, J.: Domain adaptation via pseudo in-domain data selection. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 355–362. Association for Computational Linguistics (2011)Google Scholar
  2. 2.
    Brown, P.F., Pietra, V.J.D., Pietra, S.A.D., Mercer, R.L.: The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics 19(2), 263–311 (1993)Google Scholar
  3. 3.
    Cettolo, M., Girardi, C., Federico, M.: Wit3: Web inventory of transcribed and translated talks. In: Proceedings of the 16th Conference of the European Association for Machine Translation (EAMT), pp. 261–268 (2012)Google Scholar
  4. 4.
    Dyer, C., Chahuneau, V., Smith, N.A.: A simple, fast, and effective reparameterization of IBM model 2. In: HLT-NAACL, pp. 644–648. Citeseer (2013)Google Scholar
  5. 5.
    Foster, G., Goutte, C., Kuhn, R.: Discriminative instance weighting for domain adaptation in statistical machine translation. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 451–459. Association for Computational Linguistics (2010)Google Scholar
  6. 6.
    Gao, J., Goodman, J., Li, M., Lee, K.F.: Toward a unified approach to statistical language modeling for chinese. ACM Transactions on Asian Language Information Processing (TALIP) 1(1), 3–33 (2002)CrossRefGoogle Scholar
  7. 7.
    Hildebrand, A.S., Eck, M., Vogel, S., Waibel, A.: Adaptation of the translation model for statistical machine translation based on information retrieval. In: Proceedings of EAMT, vol. 2005, pp. 133–142 (2005)Google Scholar
  8. 8.
    Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., et al.: Moses: Open source toolkit for statistical machine translation. In: Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, pp. 177–180. Association for Computational Linguistics (2007)Google Scholar
  9. 9.
    Koehn, P., Schroeder, J.: Experiments in domain adaptation for statistical machine translation. In: Proceedings of the Second Workshop on Statistical Machine Translation, pp. 224–227. Association for Computational Linguistics (2007)Google Scholar
  10. 10.
    Koehn, P., Senellart, J.: Convergence of translation memory and statistical machine translation. In: Proceedings of AMTA Workshop on MT Research and the Translation Industry, pp. 21–31 (2010)Google Scholar
  11. 11.
    Lü, Y., Huang, J., Liu, Q.: Improving statistical machine translation performance by training data selection and optimization. In: EMNLP-CoNLL, pp. 343–350. Citeseer (2007)Google Scholar
  12. 12.
    Leveling, J., Ganguly, D., Dandapat, S., Jones, G.J.: Approximate sentence retrieval for scalable and efficient example-based machine translation (2012)Google Scholar
  13. 13.
    Levenshtein, V.I.: Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady 10, 707 (1966)MathSciNetGoogle Scholar
  14. 14.
    Lin, S.C., Tsai, C.L., Chien, L.F., Chen, K.J., Lee, L.S.: Chinese language model adaptation based on document classification and multiple domain-specific language models. In: Fifth European Conference on Speech Communication and Technology (1997)Google Scholar
  15. 15.
    Lu, Y., Wang, L., Wong, D.F., Chao, L.S., Wang, Y., Oliveira, F.: Domain adaptation for medical text translation using web resources. In: ACL 2014, p. 233 (2014)Google Scholar
  16. 16.
    Moore, R.C., Lewis, W.: Intelligent selection of language model training data. In: Proceedings of the ACL 2010 Conference Short Papers, pp. 220–224. Association for Computational Linguistics (2010)Google Scholar
  17. 17.
    Och, F.J.: Minimum error rate training in statistical machine translation. In: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, vol. 1, pp. 160–167. Association for Computational Linguistics (2003)Google Scholar
  18. 18.
    Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)Google Scholar
  19. 19.
    Socher, R., Pennington, J., Huang, E.H., Ng, A.Y., Manning, C.D.: Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 151–161. Association for Computational Linguistics (2011)Google Scholar
  20. 20.
    Stolcke, A.: others: SRILM-an extensible language modeling toolkit. In: INTERSPEECH (2002)Google Scholar
  21. 21.
    Tian, L., Wong, D., Chao, L., Quaresma, P., Oliveira, F., Lu, Y., Li, S., Wang, Y., Wang, L.: UM-corpus: a large english-chinese parallel corpus for statistical machine translation. In: Proceedings of the 9th International Conference on Language Resources and Evaluation (2014)Google Scholar
  22. 22.
    Wang, L., Lu, Y., Wong, D.F., Chao, L.S., Wang, Y., Oliveira, F.: Combining domain adaptation approaches for medical text translation. In: ACL 2014, p. 254 (2014)Google Scholar
  23. 23.
    Wang, L., Wong, D.F., Chao, L.S., Lu, Y., Xing, J.: A systematic comparison of data selection criteria for SMT domain adaptation. The Scientific World Journal 2014 (2014)Google Scholar
  24. 24.
    Wang, L., Wong, D.F., Chao, L.S., Xing, J.: CRFs-based chinese word segmentation for micro-blog with small-scale data. In: Proceedings of the Second CIPS-SIGHAN Joint Conference on Chinese Language, pp. 51–57. Citeseer (2012)Google Scholar
  25. 25.
    Wang, L., Wong, D.F., Chao, L.S., Xing, J., Lu, Y., Trancoso, I.: Edit distance: A new data selection criterion for domain adaptation in SMT. In: RANLP, pp. 727–732 (2013)Google Scholar
  26. 26.
    Yasuda, K., Zhang, R., Yamamoto, H., Sumita, E.: Method of selecting training data to build a compact and efficient translation model. In: IJCNLP, pp. 655–660 (2008)Google Scholar
  27. 27.
    Zhao, B., Eck, M., Vogel, S.: Language model adaptation for statistical machine translation with structured query models. In: Proceedings of the 20th International Conference on Computational Linguistics, p. 411. Association for Computational Linguistics (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Yi Lu
    • 1
  • Derek F. Wong
    • 1
  • Lidia S. Chao
    • 1
  • Longyue Wang
    • 1
  1. 1.Natural Language Processing & Portuguese-Chinese Machine Translation Laboratory, Department of Computer and Information ScienceUniversity of MacauMacauChina

Personalised recommendations