Abstract
The problem of record linkage is a widely studied problem that aims to identify coreferent (i.e. duplicate) data in a structured data source. As indicated by Winkler, a solution to the record linkage problem is only possible if the error rate is sufficiently low. In other words, in order to successfully de-duplicate a database, the objects in the database must be of sufficient quality. However, this assumption is not always feasible. In this paper, it is investigated how merging of low quality objects into one high quality object can improve the process of record linkage. This general idea is illustrated in the context of strings comparison, where strings of low quality (i.e. with a high typographical error rate) are merged into a string of high quality by using an n-dimensional Levenshtein distance matrix and compute the optimal alignment between the dirty strings. Results are presented and possible refinements are proposed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bilenko, M., Mooney, R.J.: Learning to combine trained distance metrics for duplicate detection in databases. Tech. rep. (2002)
Bronselaer, A., Hallez, A., De Tré, G.: Extensions of fuzzy measures and the sugeno integral for possibilistic truth values. International Journal of Intelligent Systems 24(2), 97–117 (2009)
Fellegi, I., Sunter, A.: A theory for record linkage. American Statistical Association Journal 64(328), 1183–1210 (1969)
Gusfield, D.: Algorithms on strings, trees, and sequences: computer science and computational biology. Cambridge University Press, New York (1997)
Lehti, P., Fankhauser, P.: Probabilistic Iterative Duplicate Detection. In: Meersman, R. (ed.) OTM 2005. LNCS, vol. 3761, pp. 1225–1242. Springer, Heidelberg (2005)
Levenshtein, V.I.: Binary codes capable of correcting deletions, insertions, and reversals. Tech. Rep. 8 (1966)
Ravikumar, P., Cohen, W.W.: A hierarchical graphical model for record linkage. In: Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, UAI 2004, pp. 454–461. AUAI Press, Arlington (2004)
Tejada, S., Knoblock, C.A., Minton, S.: Learning object identification rules for information integration. Information Systems 26, 607–633 (2001)
Winkler, W.E.: Methods for record linkage and bayesian networks. Tech. rep., Series RRS2002/05, U.S. Bureau of the Census (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nielandt, J., Bronselaer, A., De Tré, G. (2012). Coreference Detection of Low Quality Objects. In: Greco, S., Bouchon-Meunier, B., Coletti, G., Fedrizzi, M., Matarazzo, B., Yager, R.R. (eds) Advances on Computational Intelligence. IPMU 2012. Communications in Computer and Information Science, vol 297. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31709-5_46
Download citation
DOI: https://doi.org/10.1007/978-3-642-31709-5_46
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31708-8
Online ISBN: 978-3-642-31709-5
eBook Packages: Computer ScienceComputer Science (R0)