Joint deduplication of multiple record types in relational data

Aron Culotta, Andrew McCallumCIKM 2005 · OpenReview

Abstract

Record deduplication is the task of merging database records that refer to the same underlying entity. In relational data-bases, accurate deduplication for records of one type is often dependent on the decisions made for records of other types. Whereas nearly all previous approaches have merged records of different types independently, this work models these inter-dependencies explicitly to collectively deduplicate records of multiple types. We construct a conditional random field model of deduplication that captures these relational dependencies, and then employ a novel relational partitioning algorithm to jointly deduplicate records. For two citation matching datasets, we show that collectively deduplicating paper and venue records results in up to a 30% error reduction in venue deduplication, and up to a 20% error reduction in paper deduplication.
BibTeX
@inproceedings{culotta2005joint,
  author = {Aron Culotta and Andrew McCallum},
  title = {{Joint deduplication of multiple record types in relational data}},
  booktitle = {ACM International Conference on Information and Knowledge Management},
  year = {2005},
  url = {https://openreview.net/forum?id=H5IqsNgKIZ5},
}

Loading related content...

Figure