Abstract
Evaluating the quality of counterfactual explanations (CFEs) in Explainable AI (XAI) typically necessitates resource-intensive human user studies. This work introduces CounterEval, a novel dataset of human-rated CFEs across eight qualitative metrics. We demonstrate that fine-tuned Large Language Models (LLMs) can effectively serve as proxies for human evaluators, achieving up to 85% accuracy in predicting human judgments. This approach offers a scalable, automated framework for human-centric AI evaluation.
Introduction
In the field of Explainable AI (XAI), counterfactual explanations (CFEs) have emerged as a preferred method because they align closely with human causal reasoning. A CFE provides actionable feedback by specifying the minimal changes required to alter a model's output (e.g., "If your credit score were 50 points higher, your loan would be approved").
Despite their intuitive appeal, the XAI community lacks a "gold standard" for evaluating the effectiveness of these explanations. Traditional mathematical metrics, such as proximity (the magnitude of change) and sparsity (the number of features changed), often fail to account for human-centric "explanatory virtues" like feasibility, fairness, or trust. While human user studies remain the ideal evaluation method, they are difficult to scale, costly, and often yield inconsistent results. This research explores whether Large Language Models (LLMs) can bridge this gap by acting as automated proxies for human evaluators.
The CounterEval Dataset
To facilitate this study, the authors developed CounterEval, a specialized dataset designed to capture human perceptions of counterfactual quality.
The dataset consists of:
- 30 Unique Scenarios: Based on real-world benchmarks (Adult census and Pima Indians Diabetes datasets), these scenarios were engineered to vary across dimensions like feasibility and complexity.
- Human Annotations: Responses were collected from 206 participants, providing a robust ground truth.
- Multi-Metric Ratings: Each scenario was rated on a three-point scale ("low", "medium", "high") across eight metrics: Satisfaction, Feasibility, Consistency, Completeness, Trust, Understandability, Fairness, and Complexity.
Methodology
The researchers implemented a multi-stage pipeline to automate the evaluation process. The core objective was to determine if an LLM could be trained to predict the average human rating for a given explanation.
The pipeline involved benchmarking several state-of-the-art models, including GPT-4, Llama 3 (8B and 70B), and Llama 3.1 (8B). To enhance performance, the Llama models underwent fine-tuning using QLoRA (Quantized Low-Rank Adaptation) on the human-labeled data. Beyond aggregate modeling, the study also explored individual modeling by fine-tuning LLMs to mimic the specific preferences of distinct user subgroups identified through clustering.
Key Results and Insights
The experimental results demonstrate the significant potential of LLMs as evaluators:
- High Predictive Accuracy: The fine-tuned Llama 3 70B model achieved 85% accuracy in matching human labels, significantly outperforming zero-shot baselines (where GPT-4 reached 63%).
- Reliability: The fine-tuned models exhibited high consistency, rarely making "egregious" errors, such as misclassifying a "low" rating as "high."
- Metric Sensitivity: The models found metrics like Understandability and Satisfaction easier to predict. In contrast, Feasibility and Trust proved more challenging, likely because they require a deeper understanding of real-world constraints and socio-technical context.
- Personalized Evaluation: A notable finding was the model's ability to adapt to individual preferences. When trained on a specific user's historical ratings, the LLM could predict that individual's future judgments with approximately 90% accuracy.
Discussion and Limitations
While the results are promising, the authors identify several limitations. The current dataset, while rich in annotations, is limited to 30 unique scenarios. Expanding the diversity of these scenarios is crucial for broader generalizability. Furthermore, there is a risk of "gaming" the system; if developers optimize XAI models solely to satisfy an LLM evaluator, the resulting explanations might appear high-quality to the AI while failing to provide genuine utility to humans.
Conclusion
This research presents a significant step toward unifying the evaluation of counterfactual explanations. By leveraging the latent reasoning capabilities of LLMs and fine-tuning them on human-centric data, the authors provide a scalable alternative to traditional user studies. Future work will focus on integrating these automated evaluators into the CFE generation process, creating a "human-in-the-loop" system that refines explanations in real-time.
Summary updated