Abstract
Many data management applications, such as setting up Web portals, managing enterprise data, managing community data, and sharing scientific data, require integrating data from multiple sources. Each of these sources provides a set of values and different sources can often provide conflicting values. To present quality data to users, it is critical to resolve conflicts and discover values that reflect the real world; this task is called data fusion. This paper describes a novel approach that finds true values from conflicting information when there are a large number of sources, among which some may copy from others. We present a case study on real-world data showing that the described algorithm can significantly improve accuracy of truth discovery and is scalable when there are a large number of data sources.
Introduction
When multiple sources disagree, how do we find the truth? While simple majority voting is the intuitive first step, it fails spectacularly when sources are inaccurate or, worse, when they simply copy each other. This paper addresses the fundamental challenge of "data fusion"—identifying true information by simultaneously estimating source accuracy and detecting copying relationships.
The Data Fusion Problem
In fields ranging from enterprise data management to web-scale information extraction, integrating data from multiple sources often reveals conflicting values. For instance, different websites might list different authors for the same book or different affiliations for the same researcher.
The standard approach is "naive voting," where the value reported by the most sources is accepted as true. However, this is vulnerable to two major issues:
- Varying Trustworthiness: Some sources are inherently more accurate than others.
- The "Echo Chamber" Effect: If multiple websites copy a single incorrect source, naive voting will incorrectly favor the false value because of its frequency.
The authors tackle the "chicken-and-egg" problem: to know which source is accurate, you need to know the truth; but to find the truth, you need to know which sources are accurate. Furthermore, one must identify "copiers" to avoid over-counting redundant information.
Methodology: The ACCUCOPY Algorithm
The paper introduces a novel Bayesian approach to resolve these conflicts through several key mechanisms:
Bayesian Truth Discovery
Instead of simple counts, the model calculates the probability of a value being true based on the estimated accuracy of its providers.
Copy Detection via Shared Errors
This is the paper's key innovation. The authors argue that while two independent sources might easily share the same true value, it is highly improbable they would independently provide the exact same false value. Therefore, shared mistakes serve as a "smoking gun" for copying.
Iterative Refinement
Since accuracy and copying are interdependent, the ACCUCOPY algorithm is iterative. It starts with an initial guess, detects potential copying, updates the "confidence" of values, refines the accuracy scores of sources, and repeats until the results stabilize.
Directional Copying
The model uses accuracy as a clue for the direction of copying. If two sources share many values but one is significantly more accurate overall, the less accurate one is likely the copier.
Experimental Evaluation
The authors evaluated their methods using a real-world dataset of computer science book listings from AbeBooks.com, involving 877 bookstores and over 24,000 author name listings. A "gold standard" of 100 verified books was used for performance measurement.
The experimental results were striking:
- Precision Boost: Naive voting achieved 71% precision. By considering source accuracy and copying (ACCUCOPY), precision jumped to 87%.
- Similarity Modeling: When extended to account for string similarity (e.g., "J. Doe" vs. "John Doe"), the precision reached 89%.
- Identifying Hubs: The model successfully identified large bookstores that had low accuracy because they frequently propagated errors from other sources, while smaller, specialized sources were often more independent and accurate.
Conclusion and Future Work
This work provides a mathematically rigorous way to navigate the "echo chambers" of the internet. However, limitations remain. The current model assumes a uniform distribution of false values, which may not account for common typos. Additionally, it focuses on static data snapshots and does not yet handle "mutual copying" where sources continuously feed off each other. Future work may address these temporal and dependency complexities.
Summary updated