RobustX: Robust Counterfactual Explanations Made Easy

Junqi Jiang, Luca Marzari, Aaryan Purohit, Francesco Leofante

Introduction

Counterfactual Explanations (CEs) have emerged as a cornerstone of Explainable AI (XAI), offering human-interpretable insights into "black-box" machine learning models. By suggesting minimal feature changes to alter a model's prediction (e.g., "If your income were $5,000 higher, your loan would have been approved"), CEs provide actionable recourse for users. However, a significant challenge persists: fragility. A minor update to a model or a slight shift in data distribution can invalidate previously generated explanations, leading to a loss of user trust and practical utility.

The robustness problem: In (a), the red cross is a valid counterfactual. In (b), a slight shift in the decision boundary renders the same counterfactual invalid.

Despite the proposal of various "robust" CE methods, the field remains fragmented due to inconsistent codebases, datasets, and evaluation metrics. RobustX is introduced as an open-source Python library designed to standardize the generation and evaluation of robust CEs, providing a unified workbench for researchers and practitioners.

The RobustX Framework

RobustX standardizes the XAI workflow through a modular three-part pipeline:

  1. Task Wrapper: Integrates datasets with models from popular frameworks like PyTorch, Keras, and Scikit-Learn.
  2. CE Generator: Houses a library of algorithms, including nine robust methods (e.g., PROPLACE, RNCE) and four non-robust baselines (e.g., Wachter).
  3. CE Evaluator: Provides a comprehensive suite of metrics to assess validity, proximity, and robustness against model shifts or input noise.

The library includes pre-loaded datasets common in XAI research, such as the Ionosphere dataset, and provides a DataLoader for custom CSV imports. This modularity allows for seamless "plug-and-play" experimentation, enabling researchers to benchmark new algorithms against established baselines under identical conditions.

Key Insights in Robust Counterfactuals

The development and testing of RobustX have highlighted several critical trade-offs in the field:

  • Proximity vs. Robustness: There is an inherent trade-off between the "minimal change" principle and explanation reliability. Robust counterfactuals often reside deeper within the target class's decision region, necessitating larger feature changes (lower proximity) to ensure they remain valid despite model perturbations.
  • Model Multiplicity: Robustness must account for "model multiplicity," where different models trained on the same data may yield conflicting explanations. RobustX includes specific evaluators to verify CE validity across multiple potential model candidates.
  • Standardization as a Catalyst: The primary barrier to progress in robust XAI has been the lack of shared infrastructure. RobustX addresses this by enabling "apples-to-apples" comparisons that were previously difficult to perform.

Experimental Results

To demonstrate the utility of the library, the authors benchmarked several methods on the Ionosphere dataset. The results underscore the necessity of robust methods in dynamic environments.

Benchmarking results on the Ionosphere dataset. Non-robust methods (MCE) fail under model shifts, while robust methods like RNCE and PROPLACE maintain 100% validity at the cost of higher proximity or computation time.

As shown in the table, non-robust baselines like MCE achieve 0% robustness when faced with model fine-tuning. In contrast, methods like RNCE, STCE, and PROPLACE achieve 100% robustness. Notably, RNCE provides a compelling balance, achieving full robustness with relatively low computational overhead (3.9s).

Conclusion and Future Work

RobustX provides a much-needed standardized platform for the XAI community. While currently focused on classification tasks, future iterations aim to support regression problems and integrate a wider array of state-of-the-art algorithms. By lowering the barrier to rigorous benchmarking, RobustX paves the way for more reliable and trustworthy machine learning systems.

Summary updated

Abstract

The increasing use of Machine Learning (ML) models to aid decision-making in high-stakes industries demands explainability to facilitate trust. Counterfactual Explanations (CEs) are ideally suited for this, as they can offer insights into the predictions of an ML model by illustrating how changes in its input data may lead to different outcomes. However, for CEs to realise their explanatory potential, significant challenges remain in ensuring their robustness under slight changes in the scenario being explained. Despite the widespread recognition of CEs' robustness as a fundamental requirement, a lack of standardised tools and benchmarks hinders a comprehensive and effective comparison of robust CE generation methods. In this paper, we introduce RobustX, an open-source Python library implementing a collection of CE generation and evaluation methods, with a focus on the robustness property. RobustX provides interfaces to several existing methods from the literature, enabling streamlined access to state-of-the-art techniques. The library is also easily extensible, allowing fast prototyping of novel robust CE generation and evaluation methods.
BibTeX
@inproceedings{jiang2025robustx,
  author = {Junqi Jiang and Luca Marzari and Aaryan Purohit and Francesco Leofante},
  title = {{RobustX: Robust Counterfactual Explanations Made Easy}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/1264},
}
Figure