Abstract
This paper investigates how non-experts mentally combine conflicting data visualizations, a common scenario when reading science news. Through a series of behavioral experiments with 1,166 participants, the authors found that when faced with two line charts showing different trends, viewers systematically bias their synthesized mental model toward positive relationships (if trends conflict) or weaker relationships (if trends align). These findings highlight underlying cognitive biases in visual information synthesis and offer insights for designing better data storytelling and sensemaking tools.
Introduction
Scientific research is inherently incremental, often producing findings that contradict or contextualize previous studies. The general public typically encounters these conflicting results through news articles and data visualizations. While cognitive biases in information searching (like confirmation bias) are well-documented, we know very little about how non-experts synthesize conflicting visual evidence. Without formal statistical training, laypeople rely on visual and cognitive heuristics to merge multiple charts into a single mental model. This research aims to map these heuristics and identify the systematic biases they introduce.
Methods and Experimental Design
The researchers conducted four behavioral experiments using the Prolific platform. Participants were sequentially shown two line charts displaying varying relationships (slopes categorized as high, medium, or low, and either positive or negative). After viewing the charts, participants used a custom interactive widget to draw a single line representing their synthesized conclusion.
To ensure the findings were robust, the experiments increased in ecological validity:
- Experiment 1 (Baseline): Abstract charts with no context (Variables X and Y).
- Experiment 2 (Real-world scenarios): Charts labeled with relatable contexts, such as crop yield vs. water use or quality of life vs. public spending.
- Experiment 3 (Text and Visuals): Charts embedded within full-text descriptions to simulate reading conflicting online news articles.
The researchers compared the participants' drawn lines to the mathematical average of the two original charts to measure deviation and bias.
Key Insights: The Heuristics of Synthesis
The study revealed several consistent cognitive biases that override objective mathematical averaging:
- The Positivity Bias: When charts show opposite trends (one positive, one negative), viewers do not average them to a flat line. Instead, they heavily favor the positive trend, suggesting an inherent openness to the plausibility of positive correlations.
- The Weakness Bias: When both charts show trends in the same direction, participants lean toward the less steep (weaker) slope. This effect is significantly amplified when both slopes are negative.
- The "Stronger Slope" Pull: As the discrepancy between two charts increases, participants tend to abandon averaging and anchor their mental model closer to the chart with the steeper, "stronger" slope.
Quantitative Results
The systematic nature of these biases was evident across all experiments:
- Opposite Directions: When synthesizing a positive and a negative slope, participants weighted the positive slope at ~56% in the baseline, rising to ~68% when embedded in text (Exp 3).
- Same Direction: Participants weighted the weaker slope at ~59-62%. For two negative slopes, this weighting jumped to 83% in the baseline and over 100% in some contextualized scenarios, meaning participants drew a line even flatter than the weakest stimulus.
- Consistency: These behaviors remained remarkably consistent whether the charts were abstract or realistic, suggesting a deep-seated perceptual mechanism.
Discussion and Future Work
The study demonstrates that visual synthesis is not a neutral process. However, it is limited by its focus on line charts and simple two-source scenarios. Future research should explore whether these biases generalize to other visualization formats (like bar charts) and how viewers handle larger, more complex sets of conflicting data. Additionally, incorporating uncertainty markers in visualization tools might help users mitigate these systematic biases by acknowledging the inherent conflict in the data rather than forcing a single, biased deterministic conclusion.
Summary updated