A Quality Estimation System for Hungarian

  • Zijian Győző YangEmail author
  • Andrea Dömötör
  • László János Laki
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10930)


Quality estimation is an important field of machine translation evaluation. There are automatic evaluation methods for machine translation that use reference translations created by human translators. The creation of these reference translations is very expensive and time-consuming. Furthermore, these automatic evaluation methods are not real-time and the correlation between the results of these methods and that of human evaluation is very low in the case of translations from English to Hungarian. The other kind of evaluation approach is quality estimation. These methods address the task by estimating the quality of translations as a prediction task for which features are extracted from only the source and translated sentences. In this study, we describe an English-Hungarian quality estimation system that can predict quality of translated sentences. Furthermore, using the predicted the quality scores, we combined different kinds of machine translated outputs to improve the translation accuracy. For this task, we created a training corpus. Last, but not least, using the quality estimation method we created a monolingual quality estimation system for a psycholinguistically motivated parser. In this paper we summarize our results and show some partial results of ongoing projects.


Quality estimation Machine translation 


  1. 1.
    Beck, D., Shah, K., Cohn, T., Specia, L.: SHEF-Lite: when less is more for translation quality estimation. In: Proceedings of the Workshop on Machine Translation (WMT), August 2013Google Scholar
  2. 2.
    Csendes, D., Csirik, J., Gyimóthy, T., Kocsor, A.: The szeged treebank. In: Matoušek, V., Mautner, P., Pavelka, T. (eds.) TSD 2005. LNCS (LNAI), vol. 3658, pp. 123–131. Springer, Heidelberg (2005). Scholar
  3. 3.
    Gonzalez-Agirre, A., Laparra, E., Rigau, G.: Multilingual central repository version 3.0. In: Calzolari, N., Choukri, K., Declerck, T., Dogan, M.U., Maegaard, B., Mariani, J., Odijk, J., Piperidis, S. (eds.) LREC, pp. 2525–2529. European Language Resources Association (ELRA) (2012)Google Scholar
  4. 4.
    Halácsy, P., Kornai, A., Németh, L., Sas, B., Varga, D., Váradi, T., Vonyó, A.: A Hunglish korpusz és szótár. In: III. Magyar Számítógépes Nyelvészeti Konferencia. Szegedi Egyetem (2005)Google Scholar
  5. 5.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. SIGKDD Explor. Newsl. 11(1), 10–18 (2009). Scholar
  6. 6.
    Indig, B., Vadász, N., Kalivoda, Á.: Decreasing entropy: how wide to open the window? In: Martín-Vide, C., Mizuki, T., Vega-Rodríguez, M.A. (eds.) TPNC 2016. LNCS, vol. 10071, pp. 137–148. Springer, Cham (2016). Scholar
  7. 7.
    Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E.: Moses: Open source toolkit for statistical machine translation. In: Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, ACL 2007, pp. 177–180. Association for Computational Linguistics, Stroudsburg (2007).
  8. 8.
    Laki, L.J., Yang, Z.G.: Combining machine translation systems with quality estimation. In: Computational Linguistics and Intelligent Text Processing, Budapest, Hungary (2017)Google Scholar
  9. 9.
    Novák, A., Tihanyi, L., Prószéky, G.: The MetaMorpho translation system. In: Proceedings of the Third Workshop on Statistical Machine Translation, StatMT 2008, pp. 111–114. Association for Computational Linguistics, Stroudsburg (2008).
  10. 10.
    Oravecz, C., Váradi, T., Sass, B.: The Hungarian Gigaword Corpus. In: Chair, N.C.C., et al. (eds.) Proceedings of the 9th International Conference on Language Resources and Evaluation. ELRA, Reykjavik, May 2014Google Scholar
  11. 11.
    Orosz, G., Novák, A.: PurePos 2.0: a hybrid tool for morphological disambiguation. In: RANLP 2013, pp. 539–545 (2013)Google Scholar
  12. 12.
    Prószéky, G.: Industrial applications of unification morphology. In: Proceedings of the Fourth Conference on Applied Natural Language Processing, pp. 213–214. Association for Computational Linguistics, Stuttgart, October 1994.
  13. 13.
    Recski, G., Varga, D.: A Hungarian NP Chunker. The Odd Yearbook. ELTE SEAS Undergraduate Papers in Linguistics, Budapest (2009)Google Scholar
  14. 14.
    Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of the Association for Machine Translation in the Americas, pp. 223–231 (2006)Google Scholar
  15. 15.
    Specia, L., Shah, K., de Souza, J.G., Cohn, T.: QuEst - A translation quality estimation framework. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 79–84. Association for Computational Linguistics, Sofia, August 2013.
  16. 16.
    Yang, Z.G., Laki, L.J.: Quality estimation for English-Hungarian machine translation. In: 7th Language and Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics, Poznan, Poland, pp. 170–174 (2015)Google Scholar
  17. 17.
    Yang, Z.G., Laki, L.J., Siklósi, B.: HuQ: an English-Hungarian corpus for quality estimation. In: Proceedings of the LREC 2016 Workshop - Translation Evaluation: From Fragmented Tools and Data Sets to an Integrated EcosystemGoogle Scholar
  18. 18.
    Yang, Z.G., Laki, L.J.: \(\pi \)Rate: a task-oriented monolingual quality estimation system. Int. J. Comput. Linguist. Appl. 8 (2017)Google Scholar
  19. 19.
    Yang, Z.G., Laki, L.J., Siklósi, B.: Quality estimation for English-Hungarian with optimized semantic features. In: Computational Linguistics and Intelligent Text Processing, Konya, Turkey (2016)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Zijian Győző Yang
    • 1
    • 2
    Email author
  • Andrea Dömötör
    • 1
  • László János Laki
    • 1
  1. 1.MTA-PPKE Hungarian Language Technology Research GroupBudapestHungary
  2. 2.Faculty of Information Technology and BionicsPázmány Péter Catholic UniversityBudapestHungary

Personalised recommendations