Abstract
Standard counterfactual explanations provide recourse by suggesting minimal changes to input features that would alter an AI's decision. However, these methods often prioritize mathematical proximity over human feasibility, resulting in "implausible" suggestions like changing one's age or race. This paper introduces a novel longitudinal distance metric that utilizes historical records of human change to ground explanations in reality. By comparing proposed counterfactuals to observed transitions in longitudinal datasets, the method identifies actionable recourse. Experiments on clinical and census data demonstrate that while a tension exists between plausibility and validity, the longitudinal approach effectively filters out impossible suggestions and automatically identifies immutable features.
Introduction
In the field of Explainable AI (XAI), counterfactual explanations are designed to provide users with actionable recourse. If a bank’s AI denies a loan, a counterfactual might suggest: "If your income were $10,000 higher, you would have been approved." The fundamental problem is that most algorithms search for the smallest mathematical change to flip the outcome. This often leads to suggestions that are unachievable, such as asking a 50-year-old to become 20 again. Current solutions to this problem rely on manual constraints or complex causal models, which are often unavailable or prone to bias. This work proposes a shift toward using longitudinal data—observations of the same subjects over time—to define a metric of "plausibility" based on changes actually observed in the past.
Methodology
The authors, Alexander Asemota and Giles Hooker, introduce a longitudinal distance metric. Instead of calculating the shortest geometric path, the algorithm identifies real historical changes ($D$) from longitudinal data. A proposed counterfactual change ($e - x$) is then compared to the $s$ most similar historical changes found in the data. To ensure features with different scales (like salary vs. blood pressure) are comparable, distances are normalized using the Median Absolute Deviation (MAD) of observed changes.
This metric can be implemented in two primary ways:
- Post-hoc ranking: Generating a large pool of potential explanations and sorting them by their longitudinal plausibility.
- In-loop generation: Using a genetic algorithm to evolve explanations that are naturally constrained by the longitudinal metric during the search process.
Evaluation and Datasets
The approach was evaluated using two distinct datasets:
- MIMIC-III: A real-world electronic health record dataset used to predict Acute Respiratory Failure (ARF). The longitudinal baseline was established using the change in patient vitals between hour 1 and hour 4 of admission.
- Adult-Income Dataset: A census dataset used to predict if income exceeds $50k. Since this is not naturally longitudinal, the authors simulated realistic changes—such as aging or career shifts—to create a synthetic longitudinal baseline.
Key Results and Discussion
A critical insight from this research is the inherent tension between plausibility and validity. The authors found that as you force an explanation to be more realistic (plausible), it becomes harder to find one that successfully changes the AI's decision (validity). In the MIMIC-III test, valid counterfactuals were found for all 229 patients when any feature could change, but this dropped to only 120 patients when restricted to realistic transitions.
Furthermore, the study demonstrates that standard geometric distance (L1) is a poor proxy for real-world likelihood. Points that are mathematically "close" often represent extreme outliers in terms of historical plausibility. For instance, in the MIMIC-III dataset, 74% of counterfactuals generated by standard methods suggested changes never seen in the data, often involving immutable features like religion.
By looking at longitudinal data, the metric naturally penalizes changes to features that never change in the real world (like race or birth country) without requiring the user to manually flag them. In the Adult-Income simulation, the longitudinal genetic algorithm reduced the percentage of "immutable feature changes" to 0%, whereas default proximity-based algorithms suggested impossible changes for 73-84% of individuals.
Limitations and Future Work
While effective, the method faces several hurdles. It requires longitudinal data, which is harder to collect than standard snapshot datasets. Additionally, the metric is computationally expensive due to row-wise comparisons across large matrices. Future work aims to explore clustering to speed up computation and to refine the metric so it accounts for a subject's specific starting state (e.g., a $20k salary jump is more plausible for a high-earner than a low-earner).
Summary updated