In the quest to build fair algorithms, researchers often turn to causal inference. Unlike simple statistical correlations, causal fairness asks counterfactual questions: "Would this loan applicant have been approved if their race had been different?"
The challenge is that these counterfactuals are inherently unobservable. We only see what actually happened, not what might have happened in a parallel universe. This leads to the identifiability problem: the inability to calculate a unique value for a fairness metric from the data we actually have.
Recent research has shifted from trying to find a single "correct" fairness score to a more robust approach: partial identification. Instead of a single point estimate, researchers calculate sharp bounds—the tightest possible range of values consistent with the observed data and a set of assumptions.
The Identifiability Gap
Standard fairness metrics often treat identifiability as a binary: either you can calculate the metric exactly, or you can't. Karima Makhlouf, Catuscia Palamidessi, and Sami Zhioua argue that this binary view is too restrictive. In Identifiability of Causal-based Fairness Notions: A State of the Art, they provide graphical criteria to help practitioners determine if a metric is identifiable.
When a metric is non-identifiable, it usually means the "ground truth" is hidden behind unobserved factors. For example, in personalized interventions like medical treatments, we don't know if a patient who didn't receive a drug would have been harmed by it. Nathan Kallus and Angela Zhou show in Assessing Disparate Impacts of Personalized Interventions: Identifiability and Bounds that we can still audit these systems by calculating a range of possible outcomes, even when counterfactuals are missing.
Embracing Uncertainty with Sharp Bounds
If we cannot know the exact level of bias in a model, the next best thing is to know the worst-case and best-case scenarios. This is the intuition behind sharp bounds. A bound is "sharp" if it is the most restrictive interval that remains consistent with the available data.
Elias Bareinboim, Junzhe Zhang, and Saeyoung Rho developed a Bayesian framework in Partial Identification Approach to Counterfactual Fairness Assessment to derive these bounds for "black-box" algorithms. The power of this approach is in its ability to confirm bias: if the entire calculated interval (say, a range of 0.15 to 0.25) excludes zero, a researcher can conclude with high confidence that the model is unfair, even without knowing the exact causal mechanism.
Accounting for Hidden Confounders
Most causal models rely on the "unconfoundedness" assumption—the idea that we have measured every variable that influences both the treatment and the outcome. In the real world, this is rarely true. Hidden biases (unobserved confounders) can make a fair model look unfair, or vice versa.
To address this, Dennis Frauen, Maresa Schröder, and Stefan Feuerriegel introduced a neural sensitivity framework in Causal Fairness under Unobserved Confounding: A Neural Sensitivity Framework. Their method calculates how much hidden bias would be required to invalidate a model's fairness claims. If it would take a massive, unlikely hidden variable to make the model "fair," we can reasonably conclude the model is biased.
Similarly, Miruna Oprescu, Uri Shalit, and colleagues proposed the B-Learner in B-Learner: Quasi-Oracle Bounds on Heterogeneous Causal Effects Under Hidden Confounding. This tool estimates the tightest possible intervals for treatment effects, transforming sensitivity analysis into a rigorous quantitative framework for individual-level decision-making.
New Perspectives on Counterfactuals
The field is also refining what we should be measuring. Haoxuan Li, Zhichao Jiang, and their co-authors suggest in Fairness on Principal Stratum: A New Perspective on Counterfactual Fairness that we shouldn't blindly protect everyone. Instead, they propose "principal counterfactual fairness," which focuses on individuals whose protected attributes (like race or gender) have no causal effect on the actual outcome. Because these metrics rely on joint potential outcomes, they are often non-identifiable, making the use of statistical bounds essential for auditing.
Finally, practical tools are emerging to bring these theoretical bounds to real-world data. David Watson introduced leakyIV in leakyIV: Leaky Instrumental Variables, a method that provides sharp bounds for causal effects even when the "instruments" (variables used to find causal links) are imperfect or "leaky."
By moving from point estimates to sharp bounds, the field of causal fairness is becoming more honest about the limits of data. It provides a way to make high-stakes decisions not by ignoring uncertainty, but by rigorously bounding it.