TL;DR
This paper introduces NTD-CFE, a novel model-agnostic method for generating counterfactual explanations for multivariate time-series data without needing access to the model's training data. By framing the problem as a reinforcement learning task, NTD-CFE can produce more actionable explanations that require fewer and smaller changes to the original data, even for complex, opaque models.
What Problem Does This Paper Address?
Machine learning models are increasingly used for critical decisions, but their "black box" nature makes it hard to understand why a decision was made and how to change it. Counterfactual explanations (CFEs) aim to solve this by finding the smallest change to an input that would flip the model's prediction to a desired outcome. However, existing CFE methods often struggle with:
- Data Requirements: Most need access to the model's training dataset, which is often unavailable due to privacy or proprietary concerns.
- Data Complexity: Handling multivariate time-series data (data with multiple features evolving over time) is challenging.
- Model Agnosticism: Many methods only work with differentiable models, excluding popular choices like random forests.
These limitations hinder the practical application of CFEs in real-world scenarios.
Methods
The core innovation is NTD-CFE (No-Training-Dataset Counterfactual Explanations), which leverages Reinforcement Learning (RL). Here's how it works:
- RL Formulation: The CFE generation process is framed as an RL problem where the "environment" is the predictive model being explained. The RL agent learns a policy to iteratively modify the input data.
- Action Space Reduction: Instead of searching the entire, high-dimensional time-series space, NTD-CFE reduces the search to a lower-dimensional "action space." An action consists of specifying which time step, which feature, and how much to change it.
- Model Agnosticism: NTD-CFE interacts with the predictive model as a black box, requiring only the ability to query it for predictions. This means it works with any model, regardless of its internal structure or differentiability.
- No Training Data Needed: Crucially, NTD-CFE operates directly on the specific input instance needing explanation, eliminating the need for a training dataset.
- User-Defined Constraints: Users can specify preferences for which features are actionable, immutable, or preferred, and can also incorporate causal constraints, guiding the search for more meaningful CFEs.
- Reward Function: The RL agent is rewarded for finding valid CFEs (those that achieve the desired prediction) while also being penalized for making large or numerous changes (proximity and sparsity).
Datasets
The paper evaluates NTD-CFE on nine real-world multivariate time-series datasets:
- Life Expectancy
- NATOPS
- PEMS-SF
- Heartbeat
- eRing
- Racket Sports
- Basic Motions
- Japanese Vowels
- Libras
These datasets vary in their number of time steps, features, and classes, providing a comprehensive evaluation.
Key Insights and Ideas
- RL for CFE is Powerful: Framing CFE generation as an RL problem allows for efficient exploration of the complex search space, especially for time-series data.
- No Training Data is a Game-Changer: Eliminating the need for training data significantly broadens the applicability of CFEs to scenarios where data privacy or availability is a concern.
- Actionability is Key: By minimizing the magnitude and number of changes required for a CFE, NTD-CFE produces explanations that are more practical and actionable for users.
- Flexibility in Constraints: The ability to incorporate user-defined preferences and causal constraints makes the generated CFEs more relevant and interpretable in specific contexts.
Key Results
NTD-CFE consistently outperformed four state-of-the-art model-agnostic baselines across various datasets and predictive models on key metrics:
- Proximity: NTD-CFE generated CFEs with significantly smaller changes to the original input. For example, in some cases, the proximity was 10 to 50 times lower than baselines.
- Sparsity: NTD-CFE altered fewer features to achieve the desired outcome. In some instances, sparsity was 10 to 50 times better than baselines.
- Success Rate: NTD-CFE achieved high success rates, often outperforming baselines that struggled with multivariate time-series data or required training datasets.
- Validity: NTD-CFE, like some baselines, achieved perfect validity rates by design (either producing a valid CFE or none at all).
Limitations and Future Work
- Feature Scaling: Without training data, standardizing features is difficult. The paper suggests using feature feasibility weights (Wfsib) to mitigate this, but acknowledges that more sophisticated approaches might be needed.
- Predictive Model Performance: The performance of NTD-CFE depends on the frequency with which the predictive model returns the desired prediction. It assumes this desired prediction is not sparse.
- Future Work: The authors suggest extending NTD-CFE to large language models, exploring more advanced RL algorithms, and investigating alternative search strategies beyond RL for multivariate time-series data without training datasets.
Summary updated