Advertisement

Coreference Detection of Low Quality Objects

  • Joachim Nielandt
  • Antoon Bronselaer
  • Guy De Tré
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 297)

Abstract

The problem of record linkage is a widely studied problem that aims to identify coreferent (i.e. duplicate) data in a structured data source. As indicated by Winkler, a solution to the record linkage problem is only possible if the error rate is sufficiently low. In other words, in order to successfully de-duplicate a database, the objects in the database must be of sufficient quality. However, this assumption is not always feasible. In this paper, it is investigated how merging of low quality objects into one high quality object can improve the process of record linkage. This general idea is illustrated in the context of strings comparison, where strings of low quality (i.e. with a high typographical error rate) are merged into a string of high quality by using an n-dimensional Levenshtein distance matrix and compute the optimal alignment between the dirty strings. Results are presented and possible refinements are proposed.

Keywords

object merging string merging error reduction 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bilenko, M., Mooney, R.J.: Learning to combine trained distance metrics for duplicate detection in databases. Tech. rep. (2002)Google Scholar
  2. 2.
    Bronselaer, A., Hallez, A., De Tré, G.: Extensions of fuzzy measures and the sugeno integral for possibilistic truth values. International Journal of Intelligent Systems 24(2), 97–117 (2009)zbMATHCrossRefGoogle Scholar
  3. 3.
    Fellegi, I., Sunter, A.: A theory for record linkage. American Statistical Association Journal 64(328), 1183–1210 (1969)CrossRefGoogle Scholar
  4. 4.
    Gusfield, D.: Algorithms on strings, trees, and sequences: computer science and computational biology. Cambridge University Press, New York (1997)zbMATHCrossRefGoogle Scholar
  5. 5.
    Lehti, P., Fankhauser, P.: Probabilistic Iterative Duplicate Detection. In: Meersman, R. (ed.) OTM 2005. LNCS, vol. 3761, pp. 1225–1242. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  6. 6.
    Levenshtein, V.I.: Binary codes capable of correcting deletions, insertions, and reversals. Tech. Rep. 8 (1966)Google Scholar
  7. 7.
    Ravikumar, P., Cohen, W.W.: A hierarchical graphical model for record linkage. In: Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, UAI 2004, pp. 454–461. AUAI Press, Arlington (2004)Google Scholar
  8. 8.
    Tejada, S., Knoblock, C.A., Minton, S.: Learning object identification rules for information integration. Information Systems 26, 607–633 (2001)zbMATHCrossRefGoogle Scholar
  9. 9.
    Winkler, W.E.: Methods for record linkage and bayesian networks. Tech. rep., Series RRS2002/05, U.S. Bureau of the Census (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Joachim Nielandt
    • 1
  • Antoon Bronselaer
    • 1
  • Guy De Tré
    • 1
  1. 1.Document, Database and Content ManagementGhent UniversityBelgium

Personalised recommendations