Abstract
Sentence alignment is an enabling technology which extracts mass of bilingual corpora automatically from the vast and ever-growing Web pages. In this paper, we propose a novel graph-based sentence alignment approach. Compared with the existing approaches, ours is more resistant to the noise and structure diversity of Web pages by considering sentence structural features. We formulate sentence alignment to be a matching problem between nodes (bilingual sentences) of a bipartite graph. The maximum-weighted bipartite graph matching algorithm is first applied to sentence alignment for global optimal matching. Moreover, sentence merging and aligned sentence pattern detection are used to deal with the many-to-many matching issue and the low probability of aligned sentences with few mutual translated words issue respectively. We achieve good precision over 85% and recall over 80% on manually annotated data and 1 million aligned sentence pairs with over 82% accuracy are extracted from 0.8 million bilingual pages.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ahuja, R., Magnanti, T., Orlin, J.: Network flows: theory, algorithms, and applications (1993)
Braune, F., Fraser, A.: Improved unsupervised sentence alignment for symmetrical and asymmetrical parallel corpora. In: Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pp. 81–89. Association for Computational Linguistics (2010)
Chen, J., Nie, J.: Parallel web text mining for cross-language IR. In: Proceedings of RIAO 2000: Content-Based Multimedia Information Access, vol. 1, pp. 62–78 (2000)
Chuang, T., Yeh, K.: Aligning parallel bilingual corpora statistically with punctuation criteria. Computational Linguistics and Chinese Language Processing 10(1), 95–122 (2005)
Gale, W., Church, K.: A program for aligning sentences in bilingual corpora. Computational Linguistics 19(1), 75–102 (1993)
Jiang, L., Yang, S., Zhou, M., Liu, X., Zhu, Q.: Mining bilingual data from the web with adaptively learnt patterns. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL, pp. 870–878. Association for Computational Linguistics (2009)
MartÃnez, R., Abaitua, J., Casillas, A.: Bitext correspondences through rich mark-up. In: Proceedings of the 17th International Conference on Computational Linguistics, vol. 2, pp. 812–818. Association for Computational Linguistics (1998)
Munteanu, D.S., Marcu, D.: Improving machine translation performance by exploiting non-parallel corpora. Computational Linguistics 31, 477–504 (2005)
Resnik, P., Smith, N.: The web as a parallel corpus (2002)
Shi, L., Niu, C., Zhou, M., Gao, J.: A DOM tree alignment model for mining parallel data from the web. In: Proceedings of the 21st International Conference on Computational Linguistics, pp. 489–496. Association for Computational Linguistics (2006)
Zhang, Y., Wu, K., Gao, J., Vines, P.: Automatic Acquisition of Chinese–English Parallel Corpus from the Web. Advances in Information Retrieval, 420–431 (2006)
Zhen, L., Sheng, L.: Aligning Bilingual Corpora Using Sentences Location Information. 2, 40 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhu, Y., Wang, H., Ouyang, X., Yu, Y. (2011). Graph-Based Bilingual Sentence Alignment from Large Scale Web Pages. In: Muñoz, R., Montoyo, A., Métais, E. (eds) Natural Language Processing and Information Systems. NLDB 2011. Lecture Notes in Computer Science, vol 6716. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22327-3_22
Download citation
DOI: https://doi.org/10.1007/978-3-642-22327-3_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22326-6
Online ISBN: 978-3-642-22327-3
eBook Packages: Computer ScienceComputer Science (R0)