Abstract
To foster the usefulness and accountability of machine learning (ML), it is essential to explain a model's decisions in addition to evaluating its performance. Counterfactual explanations (CFEs) have gained traction as a psychologically grounded approach to generate post-hoc explanations by highlighting what changes to a model's input would have altered its prediction. However, the usability of these approaches has yet to be thoroughly validated at the human level. We introduce the Alien Zoo, an engaging, web-based, game-inspired experimental framework designed to evaluate the usability of CFEs for novice users. Our proof-of-concept study demonstrates that users receiving CFEs outperform those without explanations in iterative learning tasks and report higher subjective usability. By providing the entire code and underlying data, we aim to equip the research community with a standardized tool for reproducible XAI user studies.
Introduction: The Human-Shaped Hole in XAI
The field of Explainable Artificial Intelligence (XAI) has seen an explosion of new algorithms, yet a significant gap remains: the lack of rigorous human-centric validation. Most XAI research focuses on technical elegance—optimizing for sparsity, proximity, or computational efficiency—rather than determining if a human can actually use the resulting explanation to make better decisions.
Counterfactual Explanations (CFEs) are particularly promising because they are psychologically intuitive, framing information as "what-if" scenarios (e.g., "If you had changed X, the outcome would have been Y"). Despite their theoretical appeal, CFEs are rarely tested with real users in controlled environments. Existing studies often suffer from small sample sizes or rely on "Wizard of Oz" designs where researchers manually craft explanations. To address this, we developed the Alien Zoo framework to test real-time, ML-generated explanations in a standardized, reproducible setting.
The Alien Zoo Framework
The Alien Zoo is a web-based game where participants take on the role of zookeepers for small alien creatures called "shubs." The primary objective is to maximize the growth of a pack of shubs by deciding how much of five different plants to feed them.
Behind the scenes, a Decision Tree Regression model, trained on synthetic data, determines the growth rate based on the user's input. In the experimental condition, users receive CFEs generated by the CEML toolbox. For instance, if a user’s choice leads to poor growth, the system provides a corrective signal: "Your result would have been better if you had selected 2 more of Plant A and 1 less of Plant B." This iterative loop allows researchers to observe how a user's mental model of the underlying ML system evolves over time.
Experimental Design and Datasets
A critical challenge in XAI user studies is the interference of prior domain knowledge. If a study uses a "real-world" dataset like housing prices, participants may rely on their own intuition rather than the model's explanations. To eliminate this bias, the Alien Zoo uses synthetic "alien" data. Participants have zero prior knowledge of shub biology, forcing them to rely entirely on the system's feedback.
We conducted two experiments with varying levels of complexity:
- Experiment 1: Featured a complex three-feature interdependence (e.g., Plant A is only beneficial if Plant B is also present).
- Experiment 2: Featured a simpler two-feature interdependence.
The control group in both experiments received a summary of their past choices but no "what-if" guidance, providing a baseline for natural human learning through trial and error.
Key Results: CFEs as a Learning Compass
Our results suggest that CFEs act as a powerful "compass" for users navigating complex decision spaces.
Objective Performance Boost
Participants receiving CFEs achieved significantly larger pack sizes than the control group. In the complex task (Experiment 1), the difference was particularly stark: the CFE group showed a clear upward learning trajectory, while the control group’s performance remained flat. This suggests that for complex, interdependent data, simply seeing one's own history is insufficient to "reverse-engineer" a model's logic.
Feature Identification and Speed
In post-game surveys, CFE users were significantly more accurate at identifying the "ground truth" relevant features (p < .001). Furthermore, while all users became faster over time, the CFE group reached higher accuracy without a significant cognitive load penalty in terms of decision time.
The Subjective-Objective Gap
Interestingly, we observed a gap between perceived and actual understanding. Users in the control group sometimes reported high confidence in their understanding of the system, even when their objective performance was poor. This highlights the importance of using behavioral metrics (like task success) rather than relying solely on self-reported surveys in XAI evaluation.
Conclusion and Future Directions
The Alien Zoo demonstrates that real-time, model-generated CFEs significantly enhance human learning and decision-making in ML-driven tasks. However, challenges remain. While CFEs are excellent "local" explanations for specific data points, it is still difficult for users to synthesize these snapshots into a perfect "global" mental model of the entire system.
Future research using this framework could explore the risks of confirmation bias—where users stop exploring once they find a "good enough" solution suggested by a CFE—or investigate how CFEs impact user trust when the underlying model is intentionally biased. By making the Alien Zoo open-source, we hope to foster a more rigorous, human-centered approach to XAI research.
Summary updated