When a machine learning model exhibits discriminatory bias, the standard instinct is to modify the model itself. We add regularization terms to the loss function, introduce adversarial training steps, or tweak the architecture. However, this model-centric approach introduces a major engineering headache: the fairness intervention is tightly coupled to a specific model architecture. If you switch from a linear model to a deep neural network, you have to completely redesign your fairness constraints and retrain from scratch.
Model-agnostic fairness pre-processing flips this paradigm on its head. Instead of fixing the model, it fixes the data.
The core intuition is elegant: if you can sanitize the training dataset to remove historical biases and spurious correlations before any learning occurs, you can use any standard, off-the-shelf machine learning algorithm. The downstream model naturally inherits the fairness properties of the clean data.
Model-Centric (In-Processing):
Biased Data ──> [ Fair Loss Function + Complex Model ] ──> Fair Predictions
Data-Centric (Pre-Processing):
Biased Data ──> [ Pre-Processing Filter ] ──> Clean Data ──> [ Standard Model ] ──> Fair Predictions
The Mechanics of Distribution Sanitization
Why does data pre-processing work mathematically? Standard machine learning algorithms operate on the principle of Empirical Risk Minimization (ERM)—they simply find patterns that minimize error on the training distribution they are given. If the training distribution contains a statistical dependency between a sensitive attribute (like race or gender) and the target label, the model will exploit that dependency to maximize accuracy.
To prevent this, foundational work by Flavio Calmon et al. in Optimized Pre-Processing for Discrimination Prevention and Optimized Data Pre-Processing for Discrimination Prevention formulated pre-processing as a convex optimization problem. The goal is to learn a randomized transformation that maps the original biased data distribution to a new, sanitized distribution. This transformation simultaneously balances three competing objectives:
- Group Fairness: Controlling the statistical disparity between different demographic groups.
- Individual Utility: Ensuring the transformed data point doesn't distort an individual's true attributes too much.
- Data Preservation: Maximizing the utility of the data so it remains predictive for downstream tasks.
Because standard learning algorithms are statistically consistent, training them on a dataset that has been transformed to satisfy these geometric constraints ensures that the resulting models are inherently fair, as demonstrated by Michele Donini et al. in Empirical Risk Minimization Under Fairness Constraints.
Modifying Labels vs. Transforming Features
When designing a pre-processing pipeline, practitioners face a critical choice: do you alter the input features, or do you alter the target labels?
Label Manipulation
Modifying input features can introduce a hidden danger known as deployment distribution shift. If you alter the features of your training data but cannot alter the features of real-world users during live deployment, your model will fail.
To bypass this, Lu Zhang et al. prove in Achieving Non-Discrimination in Prediction that pre-processing can be safely restricted to modifying only the training labels ($Y$). By strategically flipping labels to remove historical bias from the target variable, the model remains perfectly consistent with the original population's feature distribution ($X$).
Building on this concept, Joshua John Ward et al. introduced FairRR: Pre-Processing for Group Fairness through Randomized Response, which uses a randomized response framework to flip labels probabilistically, allowing practitioners to precisely tune the trade-off between demographic parity and predictive utility. For more complex scenarios, Ibrahim Alabdulmohsin et al. extended this label-shifting paradigm to multi-class settings in A Reduction to Binary Approach for Debiasing Multiclass Datasets.
Feature Orthogonalization
Alternatively, you can transform the feature space so that the non-sensitive data becomes mathematically independent of the sensitive attributes. Shixiang Zhu et al. achieve this in Counterfactual Fairness Through Transforming Data Orthogonal to Bias by projecting multivariate data into a subspace that is completely orthogonal to the sensitive variables.
However, blindly transforming features can accidentally scramble the relative rankings of individuals within a group. To protect individual consistency, Ivan Brugere et al. developed a method in A Canonical Data Transformation for Achieving Inter- and Within-group Fairness that maps data into a "canonical domain." This preserves the internal feature correspondence and merit-based rankings within demographic groups while establishing strict parity between them.
Repairing Structural Bias in Graphs
The model-agnostic pre-processing philosophy becomes even more powerful—and challenging—when applied to graph-structured data (such as social networks or recommendation systems). In graphs, bias does not just sit in isolated data points; it propagates along edges through the process of message passing. If a graph exhibits high homophily (where similar people connect to each other), a Graph Neural Network (GNN) will naturally amplify historical biases.
To prevent this structural propagation, pre-processing must "repair" the network topology itself before any embeddings are generated.
Biased Graph Structure ──> [ Optimal Transport / Sampling ] ──> Balanced Graph Structure ──> [ Any GNN ]
Yushun Dong tackled this in EDITS: Modeling and Mitigating Data Bias for Graph Neural Networks by using Wasserstein distance metrics to simultaneously debias both node attributes and the underlying adjacency matrix.
A recurring mechanism for structural repair is Optimal Transport (OT). Because a graph's connections can be viewed as probability distributions, OT can be used to align the connectivity profiles of different demographic groups. Charlotte Laclau et al. (All of the Fairness for Edge Prediction with Optimal Transport) and Hongyuan Zha et al. (Obtaining Dyadic Fairness by Optimal Transport) demonstrated that modifying the adjacency matrix via OT ensures that any downstream link prediction or node classification model inherits fairness properties, completely independent of the specific GNN architecture used.
Other structural pre-processing techniques alter how algorithms traverse the graph. Ahmad Khajehnejad proposed modifying random walk probabilities across demographic boundaries in CrossWalk: Fairness-Enhanced Node Representation Learning, while Xiao Lin et al. developed BeMap: Balanced Message Passing for Fair Graph Neural Network, which acts as a modular sampling layer that forces nodes to aggregate information from a demographically balanced neighborhood.
Causal and Counterfactual Data Warping
The most rigorous frontier of data pre-processing moves beyond simple statistical correlations and embraces causal inference. The goal here is counterfactual fairness: ensuring that a model's decision for an individual remains identical in a parallel universe where their sensitive attribute (e.g., race or gender) was changed.
To achieve this without modifying the model, researchers use causal graphs to rewrite or "warp" the dataset itself.
Ludwig Bothmann et al. introduced a compelling framework in Causal Fair Machine Learning via Rank-Preserving Interventional Distributions that mathematically transforms real-world observations into a fictitious, causally fair world where the sensitive attribute has zero causal effect on the outcome. By preserving an individual's relative rank during this transformation, the data remains highly useful for training standard models.
Similarly, Haoyu Chen et al. developed the FLAP algorithm in On Learning and Testing of Counterfactual Fairness through Data Preprocessing to strip causal dependencies from input features before training begins.
This causal approach is particularly effective at breaking "spurious shortcuts"—situations where a model latches onto a proxy feature (like a hospital ID code) instead of a true underlying signal (like actual medical symptoms). As shown by Shalmali Joshi et al. in Pulling Up by the Causal Bootstraps: Causal Data Augmentation for Pre-training Debiasing, using causal graphs to resample and deconfound the training distribution forces downstream models to learn robust, invariant features that generalize far better to new environments.
Why Pre-Processing Outperforms Post-Processing
Practitioners often debate whether it is more effective to clean data before training (pre-processing) or to adjust a model's predictions after training (post-processing).
Sajad Khodadadian et al. provided a definitive mathematical answer to this debate in Impact of Data Processing on Fairness in Supervised Learning. They proved that intervening at the data input stage is fundamentally more powerful than intervening at the output stage. Specifically, under most conditions, an optimal data pre-processor can perfectly replicate the results of any post-processor while achieving a strictly superior trade-off between fairness and accuracy.
The intuition behind this mathematical proof is straightforward: post-processing is a blunt instrument that alters the final outputs of a model, often destroying nuanced, non-linear relationships that the model worked hard to learn. Pre-processing, by contrast, cleans the underlying signal at the source. This allows the downstream machine learning model to use its full capacity to find complex, high-utility predictive patterns, unburdened by the need to navigate internal architectural constraints or compromised outputs.
Key Foundational Work
Optimized Pre-Processing for Discrimination Prevention (2017) — contributed 2 concept elements to this direction.
Optimized Data Pre-Processing for Discrimination Prevention. (2017) — contributed 2 concept elements to this direction.
Empirical Risk Minimization Under Fairness Constraints (2018) — contributed 1 concept elements to this direction.