The VLDB Journal

, Volume 22, Issue 6, pp 773–795

Joint entity resolution on multiple datasets

Authors

    • Google Research
  • Hector Garcia-Molina
    • Computer Science DepartmentStanford University
Regular Paper

DOI: 10.1007/s00778-013-0308-z

Cite this article as:
Whang, S.E. & Garcia-Molina, H. The VLDB Journal (2013) 22: 773. doi:10.1007/s00778-013-0308-z

Abstract

Entity resolution (ER) is the problem of identifying which records in a database represent the same entity. Often, records of different types are involved (e.g., authors, publications, institutions, venues), and resolving records of one type can impact the resolution of other types of records. In this paper we propose a flexible, modular resolution framework where existing ER algorithms developed for a given record type can be plugged in and used in concert with other ER algorithms. Our approach also makes it possible to run ER on subsets of similar records at a time, important when the full data are too large to resolve together. We study the scheduling and coordination of the individual ER algorithms, in order to resolve the full dataset, and show the scalability of our approach. We also introduce a “state-based” training technique where each ER algorithm is trained for the particular execution context (relative to other types of records) where it will be used.

Keywords

Entity resolutionJoint entity resolutionPhysical executionInfluence graphExecution planExpander functionState-based trainingData cleaning

Copyright information

© Springer-Verlag Berlin Heidelberg 2013