Abstract
Machine Translation is increasingly applied to medical cross-lingual data processing. In order to evaluate the quality of machine translation, automatic evaluation approaches like BLEU and NIST, most of which are n-gram based metrics, are widely used besides costly human evaluation. Current evaluation approaches merely make surface linguistic comparisons between the candidate and reference translations. Furthermore the domain features such medical terms, dependent and cohesive relations in and among sentences in medical documents should be taken into account when evaluating translations. However severe noises are imported into the procedure when faulty machine translations are parsed using the current syntactic parsers. Therefore using the noisy parsing results to compare with references affects the improvement of evaluation even though the deep processing is incorporated. To lessen noises as well as grasp the main meaning of a sentence, the paper proposes to extract the dependency n-grams only based on dependency parsing of reference translations. Dependency n-grams are stemmed and extended according to linguistic rules and then viewed as the key points for quality evaluation. The score of candidate translation is computed according to the count of dependency n-grams loose matching. Also the penalty of short translation and the clip count of the highest frequency of dependency n-grams are incorporated in the final score of the candidate. Experiments on our translation datasets show the evaluation based on dependency n-grams significantly outperforms the metric of BLEU and NSIT. This approach is also significantly better than the related research which employs dependency relation parsing in evaluation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, B., Cherry, C.: A systematic comparison of smoothing techniques for sentence-level BLEU. In: ACL 2014, p. 362 (2014)
Cohen, J., et al.: A coefficient of agreement for nominal scales. Educ. Psychol. Measur. 20(1), 37–46 (1960)
Doddington, G.: Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In: Proceedings of the second international conference on Human Language Technology Research, pp. 138–145 (2002)
Lin, C.Y., Och, F.J.: ORANGE: a method for evaluating automatic evaluation metrics for machine translation. In: Proceedings of the 20th International Conference on Computational Linguistics, ACL, pp. 501–507 (2004)
Liu, D., Gildea, D.: Syntactic features for evaluation of machine translation. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pp. 25–32 (2005)
Owczarzak, K., van Genabith, J., Way, A.: Labelled dependencies in machine translation evaluation. In: Proceedings of the Second Workshop on Statistical Machine Translation, pp. 104–111. Association for Computational Linguistics (2007)
Papineni, K., Roukos, S., Ward, T., et al.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL, pp. 311–318 (2002)
Sidorov, G., Velasquez, F., Stamatatos, E., Gelbukh, A., Chanona-Hernández, L.: Syntactic n-grams as machine learning features for natural language processing. Expert Syst. Appl. 41(3), 853–860 (2014)
Ye, Y., Zhou, M., Lin, C.Y.: Sentence level machine translation evaluation as a ranking problem: one step aside from BLEU. In: Proceedings of the Second Workshop on Statistical Machine Translation, pp. 240–247. Association for Computational Linguistics (2007)
Ying, L., Yumei, Z.: Lexis in chinese-english translation of drug package inserts: corpus-based error analysis and its translation strategies. Int. J. Biomed. Sci. IJBS 6(4), 344 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Qin, Y., Liang, Y. (2017). Medical Data Machine Translation Evaluation Based on Dependency N-Grams. In: Xing, C., Zhang, Y., Liang, Y. (eds) Smart Health. ICSH 2016. Lecture Notes in Computer Science(), vol 10219. Springer, Cham. https://doi.org/10.1007/978-3-319-59858-1_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-59858-1_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-59857-4
Online ISBN: 978-3-319-59858-1
eBook Packages: Computer ScienceComputer Science (R0)