In the world of tabular data—the spreadsheets of medical records, financial histories, and logistics logs—Gradient Boosted Decision Trees (GBDTs) and Random Forests are the undisputed champions. They handle missing values, categorical features, and non-linear relationships with a robustness that neural networks often struggle to match.
However, these models possess a fundamental architectural trait that makes them "fairness-unfriendly": they are non-differentiable. Because trees make decisions through discrete, step-like splits (e.g., Is age > 25?), you cannot use standard gradient descent to "nudge" the model toward a fairer outcome. This creates a technical wall for researchers trying to ensure that high-stakes algorithms—like those assigning liver transplants or approving loans—don't discriminate based on race or gender.
The Differentiability Gap
Most modern fairness interventions rely on gradients. If a model is biased, we calculate the derivative of the bias with respect to the model's weights and update the weights to reduce that bias. But as André F Cruz and colleagues point out in FairGBM: Gradient Boosting with Fairness Constraints, the tree-based structures used in GBDTs are non-differentiable. You cannot "flow" a fairness constraint back through a series of hard boolean splits.
This incompatibility has historically forced a difficult choice: use a "smooth" but potentially less accurate model (like a neural network) to get fairness, or use a high-performance tree model and accept the bias.
Smoothing the Path to Fairness
One way to bridge this gap is to change how we define the "cost" of unfairness. In FairGBM, André Ferreira Cruz and Catarina G Belém introduce a "proxy-Lagrangian" formulation. Instead of trying to differentiate the discrete tree itself, they use smooth "surrogates"—mathematical stand-ins like cross-entropy—to represent fairness constraints. This allows the training process to treat fairness as just another objective to optimize, achieving a 10x speedup over previous methods while maintaining the predictive power of GBDTs.
Hybridizing for Clinical Precision
In sensitive fields like medicine, the stakes are even higher. When predicting liver transplant graft failure, Ruixiang Tang, Na Zou, and their team noted that while trees are excellent for medical tabular data, their non-differentiable nature blocks standard debiasing.
Their solution, detailed in Fairly Predicting Graft Failure in Liver Transplant for Organ Assigning, is a two-step hybrid approach. By combining the strengths of tree-based models with neural network components, they create a framework where the predictive heavy lifting is done by the trees, but the fairness "tuning" is handled by differentiable layers. This allows the model to outperform traditional clinical scores like MELD while significantly reducing racial and gender disparities.
Surgical Fairness via Game Theory
What if you already have a trained model and you don't want to (or can't) retrain it? This is where post-processing becomes essential. However, even post-processing is difficult for non-differentiable models when the goal is "path-specific counterfactual fairness"—ensuring that a specific causal path (like a person's zip code acting as a proxy for race) isn't influencing the result.
In Path-Specific Counterfactual Fairness via Dividend Correction, Daisuke Hatano and Satoshi Hara look to cooperative game theory for an answer. They use Harsanyi dividends to decompose the model's output into contributions from different sets of features.
Think of it as a surgical strike: instead of retraining the whole model, they identify the specific "dividends" (contributions) that represent unfair causal influences and remove them from the final output. Because this method works on the model's outputs rather than its internal gradients, it provides a way to enforce rigorous fairness standards on "black-box" or non-differentiable architectures like Random Forests without requiring expensive retraining.
Why It Works
The common thread across these papers is the realization that we don't need to make the model itself differentiable to achieve fairness. We can:
- Optimize with surrogates: Use smooth mathematical proxies during training (FairGBM).
- Hybridize: Pair trees with differentiable components (Tang et al.).
- Decompose: Use game theory to surgically remove unfair influences after the model is built (Hatano & Hara).
By decoupling the fairness mechanism from the model architecture, this research direction ensures that the most powerful tools for tabular data are no longer off-limits for ethical, equitable machine learning.