In an ideal world, every new piece of data would provide a fresh perspective on the truth. If ten independent witnesses all claim a car was red, our confidence in that fact should skyrocket. However, in the messy reality of the web, crowdsourcing, and sensor networks, information sources are rarely independent. They copy each other, share underlying biases, or rely on the same flawed sensors.
When systems ignore these dependencies, they fall into the "independence trap": treating redundant or copied information as independent confirmation. This leads to overconfidence, where a single error—if repeated enough times—can masquerade as an indisputable consensus.
The Independence Trap
The core challenge in "truth discovery" is determining the most reliable information when multiple sources provide conflicting reports. Early frameworks, such as the perceptual reasoning work by Thomas D. Garvey, John D. Lowrance, and Martin A. Fischler, recognized that simply pooling probabilities is insufficient. They leveraged the Dempster-Shafer (D-S) theory of belief functions to model not just what is known, but the degree of ignorance and the potential for conflicting evidence A Framework for Evidential-Reasoning Systems.
The danger arises when these fusion mechanisms assume sources are independent. As Xuan Liu and Divesh Srivastava point out in their work on the SOLARIS system, web sources frequently propagate errors by scraping one another Online Data Fusion. If a system treats ten websites that all copied the same typo as ten independent votes for that typo, the truth is lost to the echo chamber.
Detecting the Smoking Gun of Copying
How can a system tell if two sources are independent or if one is merely a "copier"? The intuition lies in the nature of errors. Independent observers might occasionally agree on the truth, but they are unlikely to make the exact same rare mistake.
Researchers like Dalia Attia Waguih and Laure Berti-Equille have shown that analyzing patterns of shared errors is a powerful way to detect copying relationships Truth Discovery Algorithms: An Experimental Evaluation. By identifying these dependencies, systems can implement dependency-aware weight adjustment, discounting the votes of suspected copiers to prevent them from artificially inflating the confidence of a claim.
This is particularly critical in massive datasets like Linked Data. Systems like TruthDiscover, developed by Jun Liu and Wenqiang Liu, use graph-based models and Hidden Markov Random Fields to resolve conflicts while remaining vigilant against error propagation from unreliable, scale-free sources TruthDiscover: Resolving Object Conflicts on Massive Linked Data.
Modeling the Spectrum of Correlation
Dependency isn't always about malicious copying; it can be a byproduct of shared infrastructure or social circles. Guo-Jun Qi and Charu C. Aggarwal introduced Multi-Source Sensing (MSS) to automatically group dependent sources together, ensuring that "clusters" of biased sources do not dominate the final result Learning from Collective Intelligence in Groups.
Furthermore, Ravali Pochampally and colleagues expanded this by modeling a broad spectrum of both positive and negative correlations Fusing Data with Correlations. Their Bayesian framework accounts for:
- Positive Correlation: Sources that tend to agree because they share a common origin or bias.
- Negative Correlation: Specialized, complementary sources that provide unique facts, ensuring their contributions aren't discarded just because they don't match the majority.
When Consensus Becomes Suspicious
Perhaps the most counterintuitive finding in this field is that overwhelming agreement can actually be a sign of failure. Lachlan J. Gunn and Derek Abbott provided a mathematical proof showing that if there is even a tiny chance of systemic failure (like a biased test), unanimous evidence from a large number of sources can paradoxically decrease our confidence in a hypothesis Too good to be true: when overwhelming evidence fails to convince.
This "ceiling on certainty" highlights why modeling inter-source dependency is not just an optimization, but a fundamental requirement for trustworthy AI. Whether aggregating digital traces for pandemic tracking An early warning approach to monitor COVID-19 activity or classifying multi-view data in deep learning Trusted Multi-View Classification, the goal is to move from simple majority rule to a nuanced understanding of why sources agree. Only by de-duplicating redundant evidence and accounting for shared biases can we distill the true signal from the noise of the crowd.