Abstract
Translation tasks, including bilingual concordancing, demand an efficient space/time trade-off, which is not always easy to get due to the usage of huge text collections and the space consuming nature of time efficient text indexes. We propose a compact representation for monotonically aligned parallel texts, based on known compressed text indexes for representing the texts and additional uncompressed structures for the alignment. The proposed framework is able to index a collection of texts in main memory, occupying less space than the text size and with efficient query response time. The proposal supports any type of alignment granularity, a novelty in concordancing applications, allowing a flexible environment for linguistics working in all phases of a translation process. We present two alternatives for self-indexing the texts, and two alternatives for supporting the alignment, comparing the alternatives in terms of space/time performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aires, J., Lopes, G., Gomes, L.: Phrase translation extraction from aligned parallel corpora using suffix arrays and related structures. In: Lopes, L.S., Lau, N., Mariano, P., Rocha, L.M. (eds.) EPIA 2009. LNCS, vol. 5816, pp. 587–597. Springer, Heidelberg (2009)
Apostolico, A.: The myriad virtues of subword trees. In: Combinatorial Algorithms on Words. NATO ISO Series, pp. 85–96. Springer (1985)
Bilbao, V.D., Lopes, J.P., Ildefonso, T.: Measuring the impact of cognates in parallel text alignment. In: Portuguese Conference on Artificial Intelligence, EPIA 2005, pp. 338–343. IEEE (2005)
Bourdaillet, J., Huet, S., Langlais, P., Lapalme, G.: Transsearch: from a bilingual concordancer to a translation finder. Machine Translation 24(3), 241–271 (2010)
Brisaboa, N.R., Fariña, A., Navarro, G., Places, Á.S., Rodríguez, E.: Self-indexing natural language. In: Amir, A., Turpin, A., Moffat, A. (eds.) SPIRE 2008. LNCS, vol. 5280, pp. 121–132. Springer, Heidelberg (2008)
Brisaboa, N., Fariña, A., Ladra, S., Navarro, G.: Reorganizing compressed text. In: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 139–146. ACM (2008)
Callison-Burch, C., Bannard, C.: A compact data structure for searchable translation memories. In: EAMT 2005 (2005)
Gomes, L., Aires, J., Lopes, G.: Parallel texts alignment. In: Proceedings of the New Trends in Artificial Intelligence, 14th Portuguese Conference in Artificial Intelligence, EPIA 2009, Aveiro, pp. 513–524. Universidade de Aveiro (October 2009)
González, R., Grabowski, S., Mäkinen, V., Navarro, G.: Practical implementation of rank and select queries. In: Poster Proc. Volume of 4th Workshop on Efficient and Experimental Algorithms, pp. 27–38. CTI Press and Ellinika Grammata (2005)
Grossi, R., Gupta, A., Vitter, J.: High-order entropy-compressed text indexes. In: Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 841–850. Society for Industrial and Applied Mathematics (2003)
Kilgarriff, A., Rychly, P., Smrz, P., Tugwell, D.: Itri-04-08 the sketch engine. Information Technology 105, 116 (2004)
Koehn, P.: Europarl: A parallel corpus for statistical machine translation. In: MT summit. 5 (2005)
Kremer, G., Hartung, M., Padó, S., Riezler, S.: Statistical machine translation support improves human adjective translation. Translation: Computation, Corpora, Cognition 2(1) (2012)
Lopez, A.: Hierarchical phrase-based translation with suffix arrays. In: Proc. of EMNLP-CoNLL, pp. 976–985 (2007)
Macklovitch, E., Simard, M., Langlais, P.: Transsearch: A free translation memory on the world wide web. In: Second International Conference On Language Resources and Evaluation, LREC, vol. 3, pp. 1201–1208 (2000)
Manber, U., Myers, G.: Suffix arrays: a new method for on-line string searches. SIAM Journal on Computing 22(5), 935–948 (1993)
Puglisi, S.J., Smyth, W.F., Turpin, A.: Inverted files versus suffix arrays for locating patterns in primary memory. In: Crestani, F., Ferragina, P., Sanderson, M. (eds.) SPIRE 2006. LNCS, vol. 4209, pp. 122–133. Springer, Heidelberg (2006)
Sadakane, K.: New text indexing functionalities of the compressed suffix arrays. Journal of Algorithms 48(2), 294–313 (2003)
Tiedemann, J.: News from opus-a collection of multilingual parallel corpora with tools and interfaces. Recent Advances in Natural Language Processing 5, 237–248 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Costa, J., Gomes, L., Lopes, G.P., Russo, L.M.S., Brisaboa, N.R. (2013). Compact and Fast Indexes for Translation Related Tasks. In: Correia, L., Reis, L.P., Cascalho, J. (eds) Progress in Artificial Intelligence. EPIA 2013. Lecture Notes in Computer Science(), vol 8154. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40669-0_43
Download citation
DOI: https://doi.org/10.1007/978-3-642-40669-0_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40668-3
Online ISBN: 978-3-642-40669-0
eBook Packages: Computer ScienceComputer Science (R0)