Introduction
Explainable AI (XAI) aims to foster human-AI collaboration by providing justifications for model predictions. However, a growing body of research highlights a significant drawback: overreliance. When a model provides an explanation—such as highlighting keywords in a text—humans are often more likely to trust the prediction, even when it is incorrect. In practice, many current XAI methods function more as "advocates" for a decision rather than neutral observers, leading users to follow the model blindly.
This paper proposes a novel solution: "dissenting explanations." By presenting a second, conflicting explanation from an alternative model, the system acts as a "devil's advocate," encouraging users to think more critically and reducing overreliance without compromising overall decision accuracy.
Experimental Framework
To test this concept, the researchers conducted a human behavioral study using a "deceptive hotel review" task. Participants were asked to distinguish between genuine TripAdvisor reviews and fake ones. The study compared four distinct conditions:
- $C_0$ (Prediction only): The model provides only its classification.
- $C_1$ (One explanation): The model provides a prediction and a supporting LIME explanation.
- $C_2$ (Dissenting): The system provides the original prediction and explanation, alongside a "dissenting" prediction and explanation from a second model.
- $C_3$ (Both features): A single explanation showing both positive and negative features for the same prediction.
Generating Dissenting Voices
A key technical challenge is generating a plausible dissenting voice. The authors developed two primary algorithmic strategies:
- Global Disagreement: Training a second model ($g$) to maximize disagreement with a reference model ($f$) while maintaining high accuracy. This was achieved through Regularization (REG) to penalize similar predictions and Reweighting (WEIGHTS) to force the new model to focus on examples the first model misclassified.
- Local Disagreement: Techniques to "flip" the label of a specific instance by retraining on a small subset or adjusting the training data, allowing for the generation of a dissenting explanation for any specific input.
Key Results
The study yielded several critical insights into human-AI interaction:
- Significant Reduction in Overreliance: The dissenting condition ($C_2$) significantly reduced overreliance compared to the single explanation condition ($C_1$) ($p < 0.001$). As shown in the results, participant overreliance dropped from 0.61 in $C_1$ to 0.49 in $C_2$.
- Stable Decision Accuracy: While humans were more skeptical of the AI when it was wrong, they did not stop listening when it was right. Overall team accuracy remained stable at approximately 58-59% across all conditions.
- Trust Calibration: Dissenting explanations lowered self-reported trust to a more realistic level, suggesting that the "devil's advocate" approach successfully calibrated human trust rather than simply inducing general distrust.
Discussion: Explanations as Arguments
The core argument of this paper is that we should stop viewing XAI as a neutral window into a "black box" and instead recognize it as a persuasive argument. If explanations are inherently biased toward supporting a prediction, providing only one side is fundamentally misleading.
The researchers found a crucial distinction between dissent ($C_2$) and ambiguity ($C_3$). Humans were far more effective at catching AI errors when presented with two distinct "voices" than when shown a single model's internal pros and cons. This suggests that the social aspect of disagreement is a powerful tool for cognitive engagement.
Conclusion and Future Work
Dissenting explanations offer a promising path toward safer human-AI collaboration. By surfacing disagreement, we can empower users to act as effective "judges" of AI advice. Future research should explore these effects on more powerful models and diverse tasks, as well as the potential of counterfactual explanations to provide even more robust defenses against overreliance.
Summary updated