Data Fusion: Resolving Conflicts from Multiple Sources

Xin Luna Dong, Laure Berti-Équille, Divesh Srivastava

Abstract

Many data management applications, such as setting up Web portals, managing enterprise data, managing community data, and sharing scientific data, require integrating data from multiple sources. Each of these sources provides a set of values and different sources can often provide conflicting values. To present quality data to users, it is critical to resolve conflicts and discover values that reflect the real world; this task is called data fusion. This paper describes a novel approach that finds true values from conflicting information when there are a large number of sources, among which some may copy from others. We present a case study on real-world data showing that the described algorithm can significantly improve accuracy of truth discovery and is scalable when there are a large number of data sources.

Introduction

When multiple sources disagree, how do we find the truth? While simple majority voting is the intuitive first step, it fails spectacularly when sources are inaccurate or, worse, when they simply copy each other. This paper addresses the fundamental challenge of "data fusion"—identifying true information by simultaneously estimating source accuracy and detecting copying relationships.

The Data Fusion Problem

In fields ranging from enterprise data management to web-scale information extraction, integrating data from multiple sources often reveals conflicting values. For instance, different websites might list different authors for the same book or different affiliations for the same researcher.

Example of conflicting data and potential copying across multiple sources.

The standard approach is "naive voting," where the value reported by the most sources is accepted as true. However, this is vulnerable to two major issues:

  1. Varying Trustworthiness: Some sources are inherently more accurate than others.
  2. The "Echo Chamber" Effect: If multiple websites copy a single incorrect source, naive voting will incorrectly favor the false value because of its frequency.

The authors tackle the "chicken-and-egg" problem: to know which source is accurate, you need to know the truth; but to find the truth, you need to know which sources are accurate. Furthermore, one must identify "copiers" to avoid over-counting redundant information.

Methodology: The ACCUCOPY Algorithm

The paper introduces a novel Bayesian approach to resolve these conflicts through several key mechanisms:

Bayesian Truth Discovery

Instead of simple counts, the model calculates the probability of a value being true based on the estimated accuracy of its providers.

Copy Detection via Shared Errors

This is the paper's key innovation. The authors argue that while two independent sources might easily share the same true value, it is highly improbable they would independently provide the exact same false value. Therefore, shared mistakes serve as a "smoking gun" for copying.

Iterative Refinement

Since accuracy and copying are interdependent, the ACCUCOPY algorithm is iterative. It starts with an initial guess, detects potential copying, updates the "confidence" of values, refines the accuracy scores of sources, and repeats until the results stabilize.

Directional Copying

The model uses accuracy as a clue for the direction of copying. If two sources share many values but one is significantly more accurate overall, the less accurate one is likely the copier.

Experimental Evaluation

The authors evaluated their methods using a real-world dataset of computer science book listings from AbeBooks.com, involving 877 bookstores and over 24,000 author name listings. A "gold standard" of 100 verified books was used for performance measurement.

The experimental results were striking:

  • Precision Boost: Naive voting achieved 71% precision. By considering source accuracy and copying (ACCUCOPY), precision jumped to 87%.
  • Similarity Modeling: When extended to account for string similarity (e.g., "J. Doe" vs. "John Doe"), the precision reached 89%.
  • Identifying Hubs: The model successfully identified large bookstores that had low accuracy because they frequently propagated errors from other sources, while smaller, specialized sources were often more independent and accurate.

Conclusion and Future Work

This work provides a mathematically rigorous way to navigate the "echo chambers" of the internet. However, limitations remain. The current model assumes a uniform distribution of false values, which may not account for common typos. Additionally, it focuses on static data snapshots and does not yet handle "mutual copying" where sources continuously feed off each other. Future work may address these temporal and dependency complexities.

Summary updated

Abstract

Many data management applications, such as setting up Web portals, managing enterprise data, managing community data, and sharing scientific data, require integrating data from multiple sources. Each of these sources provides a set of values and different sources can often provide conflicting values. To present quality data to users, it is critical to resolve conflicts and discover values that reflect the real world; this task is called {\em data fusion}. This paper describes a novel approach that finds true values from conflicting information when there are a large number of sources, among which some may copy from others. We present a case study on real-world data showing that the described algorithm can significantly improve accuracy of truth discovery and is scalable when there are a large number of data sources.
BibTeX
@inproceedings{dong2013data,
  author = {Xin Luna Dong and Laure Berti-Équille and Divesh Srivastava},
  title = {{Data Fusion: Resolving Conflicts from Multiple Sources}},
  booktitle = {Handbook of Data Quality},
  year = {2013},
  url = {https://openreview.net/forum?id=YeN9aAcll3p},
}
Figure