A common paradigm in AI safety is to "fix" a foundation model's social biases during its initial pre-training phase. The goal is to produce a clean, neutral base model that can be safely distributed for a wide variety of downstream applications.
However, a growing body of research reveals a frustrating paradox: fairness is not a static property. Even if a language or vision model is successfully debiased during pre-training, it frequently re-acquires, amplifies, or reverts to those exact social biases the moment it is fine-tuned for a specific downstream task.
To understand why this happens, we must look at how optimization works during task adaptation, why latent biases are so persistent, and how researchers are shifting from one-time preprocessing fixes to active, dynamic constraints during fine-tuning.
The Illusion of the Clean Slate
The core vulnerability of pre-training debiasing is that it often merely masks stereotypes rather than erasing them. As demonstrated by Masahiro Kaneko et al. in [Debiasing Isn't Enough!
- on the Effectiveness of Debiasing MLMs and Their Social Biases in Downstream Tasks](/work/debiasing-isnt-enough-on-the-effectiveness-of-debiasing), models quickly re-learn social prejudices when optimized on task-specific datasets.
Machine learning models are fundamentally correlation-extracting engines. When a model is fine-tuned to maximize performance on a downstream task, it searches for any statistical shortcut that can minimize its loss function. If the downstream training data or labels contain subtle, systemic demographic skews, the model will readily exploit those stereotypical associations.
This regression happens across three primary vectors:
- Latent Persistence: Pre-training biases are deeply embedded in a model's representation space. Guangliang Liu et al. show in Towards Understanding Task-agnostic Debiasing Through the Lenses of Intrinsic Bias and Forgetfulness that the intrinsic bias level of a pre-trained model serves as a mathematical lower bound for its bias after fine-tuning. When the model is adapted to specialized domains, these latent associations resurge. In fact, Thiemo Wambsganss et al. find in Bias at a Second Glance: A Deep Dive into Bias for German Educational Peer-Review Data Modeling that fine-tuning on small, domain-specific datasets can actually intensify these latent biases beyond their original levels.
- Optimization Shortcuts: Standard fine-tuning objectives prioritize accuracy over fairness. As noted by Paul Pu Liang et al. in Towards Debiasing Sentence Representations, optimizing for downstream performance naturally incentivizes the model to leverage skewed associations in the training data as predictive signals.
- Catastrophic Forgetting of Fairness: Just as models can forget how to perform old tasks when trained on new ones, they also "forget" the task-agnostic fairness constraints imposed during their initial debiasing phase. The fine-tuning process prioritizes task-specific data patterns, overriding earlier safety alignments.
Pervasiveness Across Modalities and Tuning Methods
This bias re-acquisition is not an isolated quirk of text classification; it spans across modalities and even persists through parameter-efficient tuning methods.
In the medical domain, Muhammad Shujaat Mirza et al. audited medical imaging foundation models in How Fair are Medical Imaging Foundation Models?. They discovered that even when fine-tuning on perfectly balanced datasets, the representational biases acquired during the initial pre-training phase persist, proving that downstream data balancing alone is insufficient to guarantee subgroup fairness.
One might hypothesize that parameter-efficient fine-tuning (PEFT)—where the base model's weights are frozen and only a small set of adapter parameters are trained—would preserve the base model's debiased state. However, Xudong Han et al. demonstrate in Demographics and Democracy: Benchmarking LLMs' Gender Bias and Political Leaning in European Parliament that standard PEFT techniques still exacerbate demographic performance gaps. Because the lightweight adaptation layers are optimized solely on the downstream task, they remain highly susceptible to inheriting and magnifying the biases present in the fine-tuning data.
Shifting Defense to the Adaptation Phase
Because static, pre-training interventions fail to survive downstream adaptation, recent work focuses on debias-while-tuning frameworks. Instead of treating debiasing as a preprocessing step, these methods integrate fairness constraints directly into the fine-tuning objective.
Counterfactual Contrastive Learning
One prominent strategy is to force the model's representations to be invariant to protected attributes during tuning. In Co²PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuning, Xiangjue Dong et al. introduce a method that feeds original sentences alongside counterfactual pairs (e.g., swapping "he" with "she") into a prompt-tuning architecture.
By applying a contrastive loss, the model is penalized if its internal representations change based on demographic attributes. This mathematically prevents the model from using gender or race as a shortcut for the downstream task, ensuring fairness constraints are maintained during adaptation.
Causal Invariant Learning
Another approach tackles the root cause of shortcut learning by separating causal factors from correlation. Fan Zhou et al. propose Causal-Debias: Unifying Debiasing in Pretrained Language Models and Fine-tuning via Causal Invariant Learning, which uses causal inference to ensure the model focuses strictly on task-relevant causal features while actively ignoring non-causal, stereotypical correlations.
Unlabeled Regularization
When downstream datasets lack explicit labels for protected attributes (making counterfactual generation difficult), alternative regularization techniques are required. Shahed Masoudian et al. address this in Unlabeled Debiasing in Downstream Tasks via Class-wise Low Variance Regularization by introducing a class-wise low variance regularizer.
Standard Fine-Tuning:
Downstream Data ───► Minimize Task Loss ───► Exploits Demographic Shortcuts ───► Biased Model
Debias-While-Tuning (e.g., ADPLVR):
Downstream Data ───► Minimize Task Loss ───┐
├─► Discards Sensitive Variance ───► Fair Model
Low Variance Penalty ─┘
This mechanism minimizes the variance of embeddings within each target task category. By forcing the model to compress the representation space for each class, it strips away "noisy" sensitive features (like demographic variations) while preserving the core features necessary for classification.
Similarly, Hengrui Gu et al. demonstrate in Mitigate Extrinsic Social Bias in Pre-trained Language Models via Continuous Prompts Adjustment that adjusting continuous prompts specifically for downstream task representations is far more effective than relying on pre-trained weight adjustments alone.
Ultimately, this shift in research direction highlights that fairness cannot be treated as a one-time vaccine administered during pre-training. Because optimization naturally exploits shortcuts, fairness must be treated as an active, continuous constraint throughout the entire model lifecycle.
Key Foundational Work
Bias at a Second Glance: A Deep Dive into Bias for German Educational Peer-Review Data Modeling (2022) — contributed 4 concept elements to this direction.
Debiasing Isn't Enough! - on the Effectiveness of Debiasing MLMs and Their Social Biases in Downstream Tasks (2022) — contributed 3 concept elements to this direction.
Towards Debiasing Sentence Representations (2020) — contributed 2 concept elements to this direction.