In machine learning, we often assume that a model’s high accuracy is a proxy for having "learned" the underlying truth of a dataset. We treat the internal logic of a top-performing model as a definitive map of the relationship between features and outcomes. However, a growing body of research suggests this assumption is often false.
Instead, many datasets exhibit model multiplicity: the existence of a vast set of models that achieve near-identical predictive accuracy but rely on fundamentally different internal logic. This phenomenon, often called the Rashomon effect, suggests that for any given problem, there is not one "correct" model, but a "Rashomon set" of equally valid candidates.
The Divergence of Internal Logic
The core problem of model multiplicity is that functional equivalence in performance does not imply consistency in reasoning. As Sebastian Müller et al. demonstrate, models with nearly identical output distributions can exhibit widely divergent feature attributions. This means that two models can look at the same person, predict the same outcome, but cite entirely different reasons for doing so.
This divergence is particularly acute in complex architectures. Vivian Lai et al. found that while traditional models like SVMs often agree on feature importance, deep learning models like BERT and LSTMs can provide vastly different explanations for the same text classification task. Similarly, Michele Fontanesi et al. showed that different neural architectures for graph classification develop entirely different internal "policies" to solve the same task, despite achieving identical accuracy.
Why Explanations Become Fragile
If multiple models are equally accurate but use different features, then any explanation derived from a single model is inherently arbitrary. Marc-Etienne Brunet et al. term this explanatory multiplicity, arguing that a model’s justification for a decision may be a byproduct of random initialization or training choices rather than a robust property of the data.
This creates a significant challenge for algorithmic recourse—the advice given to users on how to change their outcome (e.g., "increase your income by $5,000 to get the loan"). Klaus Broelemann et al. prove that traditional sparse counterfactual explanations are fragile; a recommendation tailored to one model might fail completely if the underlying model is swapped for another functionally equivalent one. To address this, Vineet Rajani et al. propose using relational verification to generate counterfactuals that are guaranteed to remain valid across an entire ensemble of models.
The Rashomon Set as a Resource
While multiplicity creates uncertainty, it also offers a unique opportunity for model selection. If a large set of models performs equally well, we are free to choose the one that satisfies external constraints beyond accuracy.
- Simplicity and Interpretability: Caroline Wang et al. argue that the Rashomon effect allows practitioners to prioritize simpler, human-readable models over complex "black boxes" without sacrificing statistical validity. Lesia Semenova et al. further suggest that inherent noise in real-world data naturally expands the Rashomon set, making it highly likely that a simple, high-performing model exists.
- Stakeholder Alignment: Instead of searching for a single "perfect" explanation, the EXAGREE framework proposed by Quanling Deng et al. searches through the Rashomon set to find the model whose internal logic best aligns with a stakeholder’s specific needs or domain expertise.
- Robustness Auditing: By calculating Model Class Reliance (MCR), Gavin Smith et al. show how to identify the maximum and minimum possible reliance on a variable across all equally good models. This helps identify which features are truly essential and which are merely redundant artifacts.
The Danger of X-Hacking
The existence of model multiplicity also introduces new risks. Because many models can achieve the same accuracy with different explanations, an analyst could deliberately search for a model that produces a desired narrative—such as one that hides a biased feature. G. K. Weir et al. describe this as X-hacking (explanation hacking), demonstrating that model narratives can be "flipped" in minutes by treating explanation-seeking as an optimization problem.
Ultimately, the research into model multiplicity shifts the goal of Explainable AI. As Przemyslaw Biecek et al. argue, explanations should not just be used to justify a single model's decision, but to question and explore the entire space of valid predictive strategies. Understanding the Rashomon set allows us to move from blind trust in one model to a more robust, multi-perspective understanding of the data.
Key Foundational Work
Many Faces of Feature Importance: Comparing Built-in and Post-hoc Feature Importance in Text Classification (2019) — contributed 2 concept elements to this direction.
Visual Explanation of Evidence with Additive Classifiers (2006) — contributed 1 concept elements to this direction.