When we evaluate whether an algorithmic decision-making system is "fair," our first instinct is often to look at statistical correlations. We check if a loan approval rate, a recidivism risk score, or a medical diagnosis distribution is equal across different demographic groups. However, purely statistical measures of fairness can be deeply misleading. They capture correlations but fail to capture the underlying mechanisms that drive them.
To address this, researchers have turned to the framework of causal inference, and specifically to counterfactual fairness. Instead of asking whether a model treats different groups equally on average, counterfactual fairness asks an individual-level question: Would this specific person have received the same decision if their protected attribute (such as race or gender) had been different, holding all else constant?
While intuitively powerful, operationalizing this question introduces a profound mathematical and practical challenge known as counterfactual unobservability.
The Core Dilemma: The Unseen Alternate Reality
The fundamental problem of causal inference is that we can only ever observe what actually happened (the factual outcome); we can never observe what would have happened under different circumstances (the counterfactual outcome).
Consider an algorithm predicting recidivism risk. As Aria Khademi et al. point out, determining true causal fairness requires evaluating potential outcomes across parallel worlds. To know if a high-risk prediction for an individual was causally driven by their race, we must compare their actual prediction against the prediction they would have received if their race were different. Because we cannot rewind time and change an individual's demographic traits, this counterfactual scenario is inherently unobservable.
This lack of a "ground truth" creates an immediate obstacle for auditing and validation. As Stephen R Pfohl et al. note in the context of clinical risk prediction, we cannot simultaneously observe the same patient under different sensitive attributes. Without parallel data, standard empirical validation techniques break down. We cannot simply compute a model's counterfactual error rate because the labels required to do so do not exist in our historical datasets.
The Identifiability Bottleneck
In causal inference, a quantity is said to be identifiable if it can be uniquely computed from the observed data distribution. Counterfactual fairness metrics, however, routinely suffer from an identifiable bottleneck.
As explained by Karima Makhlouf et al., moving from simple observations to hypothetical "what-if" scenarios requires strong, often unverifiable assumptions about the data-generating process. If two completely different causal mechanisms can produce the exact same observational data, the true counterfactual fairness of a system is unidentifiable.
Observational Data ───► Can be generated by ───► Causal Model A (Fair)
└───► Causal Model B (Unfair)
Result: The true fairness metric cannot be uniquely identified.
This bottleneck manifests in several ways across different domains:
- Hidden Confounding: In real-world datasets, unobserved variables often influence both the protected attributes and the outcomes. Yongkai Wu et al. emphasize that these hidden confounders prevent researchers from calculating precise causal fairness metrics without making potentially incorrect structural assumptions.
- Personalized Interventions: When evaluating policies like job training programs or medical treatments, we face a similar gap. Nathan Kallus et al. demonstrate that standard fairness metrics (like True Positive Rate) are unidentifiable for personalized interventions because we cannot definitively categorize individuals as "responders" (those who benefit) or "non-responders" without knowing their unobserved counterfactual outcomes.
- Behavioral Insufficiency: We cannot bypass this problem simply by observing how an AI agent behaves. Alexis Bellot et al. prove that an agent's commitment to counterfactual fairness cannot be verified through external behavioral observation alone. Multiple internal causal models can produce identical actions in a training environment while maintaining vastly different levels of bias when deployed in novel contexts.
Overcoming Unobservability
Because counterfactuals cannot be directly observed, the research community has developed mathematical and algorithmic strategies to bridge the observational gap. These approaches generally fall into three categories: bounding, generative simulation, and contextual equivalence.
1. Mathematical Bounding and Sensitivity Analysis
When a precise point estimate of fairness cannot be identified, the next best alternative is to calculate the tightest possible range—or bounds—consistent with the observed data.
Instead of guessing the exact counterfactual outcomes, the authors construct optimization frameworks to find the mathematical upper and lower limits of unfairness. For instance, Yongkai Wu et al. formulate the measurement of causal fairness as a constrained optimization problem to derive tight bounds from observational data.
Similarly, Yoichi Chikahara et al. introduce a metric called the Probability of Individual Unfairness (PIU) and develop a data-estimable upper bound to minimize it during training without requiring restrictive assumptions about the data-generating process. When assumptions are necessary—such as assuming an intervention never causes harm—Nathan Kallus et al. use sensitivity analysis to evaluate how robust their fairness bounds remain if those assumptions are violated.
2. Generative Simulation
Another strategy is to use deep generative models to simulate the missing counterfactual worlds, effectively creating "counterfactual twins."
Factual Input (Patient A) ───► [ Generative Model (VAE/GAN) ] ───► Counterfactual Twin
(e.g., Group: X, Health: Y) (e.g., Group: Z, Health: Y)
Stephen R Pfohl et al. utilize Variational Autoencoders (VAEs) to approximate the latent variables that govern an individual's factual state, allowing them to generate a synthetic counterfactual counterpart for auditing. Taking this a step further, Yuchen Ma et al. introduce the Generative Counterfactual Fairness Network (GCFN), which leverages Generative Adversarial Networks (GANs) to estimate unobservable counterfactual scenarios with rigorous theoretical guarantees, ensuring predictions remain consistent across simulated demographic shifts.
3. Causal Context and Equivalence
Finally, researchers look for specific structural conditions under which unobservable counterfactual properties naturally map onto observable statistical properties.
Jacy Reese Anthis et al. demonstrate that under certain causal graph structures—such as those involving specific types of measurement error or selection bias—achieving standard, easily measurable group fairness metrics is mathematically equivalent to achieving counterfactual fairness. By identifying these contexts, practitioners can enforce counterfactual fairness using standard empirical data without needing to simulate or bound unobservable variables directly.
Summary
The shift from statistical to causal fairness represents a major conceptual advance, but it forces us to confront the reality that the data we need to verify fairness is fundamentally missing. Whether auditing clinical risk models, personalized recommendations, or recidivism predictions, the core challenge remains the same: we must reason about worlds we cannot see. By shifting the focus from exact point estimation to mathematical bounding, generative approximation, and structural analysis, current research provides the tools necessary to guarantee fairness despite the inherent unobservability of the counterfactual ground truth.
Key Foundational Work
Assessing Disparate Impact of Personalized Interventions: Identifiability and Bounds (2019) — contributed 6 concept elements to this direction.
Assessing Disparate Impacts of Personalized Interventions: Identifiability and Bounds (2019) — contributed 4 concept elements to this direction.
Algorithmic Bias in Recidivism Prediction: A Causal Perspective (Student Abstract) (2020) — contributed 1 concept elements to this direction.