Longitudinal Counterfactuals: Constraints and Opportunities

Alexander Asemota, Giles HookerarXiv 2024 · DOI · arXiv

Introduction

As machine learning models increasingly automate high-stakes decisions—from loan approvals to medical diagnoses—the need for "algorithmic recourse" has become paramount. When a model delivers a negative outcome, users deserve to know not just why it happened, but how they can change their circumstances to achieve a different result. These suggestions, known as counterfactual explanations, often fail in practice because they recommend changes that are mathematically sound but physically or socially impossible (e.g., "decrease your age by 10 years").

This research introduces a novel framework for generating longitudinal counterfactuals, leveraging historical data of individual transitions to ensure that suggested paths to recourse are grounded in real-world feasibility.

The Challenge of Realistic Recourse

Existing methods for generating counterfactuals typically rely on three flawed proxies for plausibility:

  1. User Constraints: Manually defining which features can change, which is tedious and often incomplete.
  2. Data Density: Suggesting changes that lead to "common" data points, which ignores the difficulty of the transition itself.
  3. Causal Graphs: Mapping variable interactions, which is notoriously difficult to scale to complex, high-dimensional datasets.

The authors argue that the missing ingredient is longitudinal data. By observing how individuals actually change over time, we can determine which transitions are natural and which are impossible.

The Longitudinal Distance Metric

The core contribution of this work is the Longitudinal Distance Metric. Unlike standard geometric metrics (like Euclidean distance) that measure the gap between two static points, this metric measures the distance between a proposed change and historically observed trajectories.

Specifically, the metric identifies the most similar historical transitions in a dataset and penalizes any proposed counterfactual that suggests a jump never before seen in the data. This ensures that the "path" to recourse mirrors actual human experience.

Generating Counterfactuals

This metric can be integrated directly into the generation process. The authors propose a genetic algorithm approach to evolve explanations that are both effective (flipping the model's prediction) and plausible (minimizing longitudinal distance).

Algorithm 1: Genetic Counterfactuals, showing the iterative process of evolving plausible recourse paths.

Experimental Evaluation

The researchers evaluated their approach using two distinct datasets: MIMIC-III (clinical ICU data) and Adult-Income (census data).

In the clinical context, the metric successfully distinguished between "vital signs," which fluctuate naturally, and "administrative features," which remain static. In the Adult-Income dataset, where the authors simulated longitudinal changes, the results were striking. Traditional methods suggested changes to immutable features (like race or sex) for over 70% of individuals. The longitudinal method reduced these impossible suggestions to 0%.

Comparison of Longitudinal and Geometric distances (B, C) and the resulting proportions of valid recourse (A). The plots illustrate how the longitudinal metric captures realistic transitions that geometric distance misses.

The Plausibility-Validity Trade-off

A key insight from the study is the inherent tension between plausibility and validity. In biased datasets, a model might only grant a positive prediction to a specific demographic. If a user belongs to a marginalized group, the only "valid" counterfactual the model recognizes might involve changing an immutable characteristic.

When the longitudinal method forbids these impossible changes, it may find that some individuals have no valid path to recourse at all. In the Adult-Income experiment, the success rate for finding recourse dropped from 93% (default) to 53% (longitudinal). While this appears to be a drop in performance, the authors argue it is a more honest reflection of systemic barriers. It highlights that the problem often lies within the biased model or the underlying social structures, rather than the explanation method itself.

Conclusion

Longitudinal counterfactuals shift the focus of algorithmic recourse from "destinations" to "paths." By grounding suggestions in historical transitions, this approach provides a more realistic and ethical framework for human-AI interaction. While data scarcity and computational complexity remain hurdles, this work provides a vital roadmap for making algorithmic explanations truly actionable for real people.

Summary updated

Abstract

Counterfactual explanations are a common approach to providing recourse to data subjects. However, current methodology can produce counterfactuals that cannot be achieved by the subject, making the use of counterfactuals for recourse difficult to justify in practice. Though there is agreement that plausibility is an important quality when using counterfactuals for algorithmic recourse, ground truth plausibility continues to be difficult to quantify. In this paper, we propose using longitudinal data to assess and improve plausibility in counterfactuals. In particular, we develop a metric that compares longitudinal differences to counterfactual differences, allowing us to evaluate how similar a counterfactual is to prior observed changes. Furthermore, we use this metric to generate plausible counterfactuals. Finally, we discuss some of the inherent difficulties of using counterfactuals for recourse.
BibTeX
@misc{asemota2024longitudinal,
  author = {Alexander Asemota and Giles Hooker},
  title = {{Longitudinal Counterfactuals: Constraints and Opportunities}},
  year = {2024},
  eprint = {2403.00105},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.00105},
}
Figure