Algorithmic Bias Induced by Machine Unlearning

20 concepts10 papers1 proposal

The "Right to be Forgotten," formalized by regulations like GDPR, has driven the development of machine unlearning—techniques that allow organizations to erase the influence of specific training data points without the prohibitive cost of retraining a model from scratch.

While machine unlearning successfully addresses privacy concerns, it introduces a hidden, systemic side effect: it frequently degrades algorithmic fairness. When a model forgets specific data, it does not just erase a set of points; it alters its internal decision boundaries, often to the detriment of underrepresented or highly active demographic groups.

The Core Mechanism: Non-Uniform Deletion as Distribution Shift

To understand why unlearning induces bias, we must look at who requests data deletion. Standard unlearning evaluations often assume that deletion requests occur uniformly at random across a dataset. In practice, privacy preferences are highly correlated with demographic, cultural, or behavioral groups.

When deletion requests are concentrated within a specific sub-population, the training distribution undergoes a localized shift. As De Min et al. demonstrate, treating non-uniform data removal as a distribution shift problem reveals a stark reality: when a minority group disproportionately exercises privacy rights, the model's performance on the remaining members of that same group degrades significantly.

Original Balanced Data ───> [ Disproportionate Deletions from Group A ] ───> Skewed Distribution ───> Degraded Fairness for Group A

This phenomenon creates an algorithmic exclusion loop. As explored by Zhang et al., targeted deletions cause the model's training distribution to diverge from the true population distribution. Oesterling et al. note that standard unlearning techniques completely ignore the group-level comparisons required by fairness metrics. Consequently, erasing data from a minority subgroup breaks down existing fairness constraints, widening performance gaps and amplifying algorithmic disparity.

Structural Vulnerabilities: Sharding and Selection Bias

The bias induced by unlearning is not solely a product of user behavior; it is also baked into the very architectures designed to make unlearning efficient.

The Sharding Penalty

To achieve "exact unlearning" without full retraining, systems often use data sharding (such as the SISA framework), where a model is split into an ensemble of sub-models trained on distinct partitions of data. However, in recommender systems, user behavior is naturally clustered.

As highlighted in a survey by Chen et al., sharding strategies inadvertently isolate active users into specific partitions. When these active users request data removal, the corresponding shards undergo massive performance drops. Chen et al. show that this structural partitioning disproportionately optimizes the system for high-activity groups at the expense of inactive ones, severely compromising recommendation fairness.

Anticipatory Selection Bias

Organizations attempting to mitigate the cost of future deletions may turn to proactive data selection. For instance, frameworks might choose to train only on data points that have a low probability of being deleted in the future.

However, Sim et al. demonstrate that this "deletion-anticipative" strategy introduces a severe selection bias. If a demographic group is statistically more likely to request data deletion, an anticipatory algorithm will proactively underrepresent that group in the initial training set, encoding bias into the model before a single unlearning request is even processed.

Long-Term Dynamics and Malicious Exploitation

The interaction between unlearning and bias is not static; it evolves over time and can be weaponized.

  • The Feedback Loop: In deployed environments, model bias and user retention are deeply linked. Chen et al. model this as an optimal control problem, showing that initial model bias reduces minority user retention. This drop in retention leads to fewer active minority users, which yields less data, ultimately compounding the model's bias in subsequent unlearning and update cycles.
  • Malicious Forgetting Attacks: The sensitivity of fairness metrics to unlearning creates a security vulnerability. Zhao et al. established that adversaries can execute "malicious forgetting requests." By strategically requesting the erasure of specific, highly influential data points (such as core nodes in a graph or key minority samples), an attacker can intentionally collapse a model's fairness guarantees or steer it toward a biased state.

Emerging Horizons

The field is currently transitioning from identifying these vulnerabilities to engineering robust solutions. While domains like generative modeling still lack a mature understanding of these fairness implications—as noted by Wu et al. in the context of diffusion models—and decentralized networks face steep scaling hurdles (Shaik et al.), explicit algorithmic remedies are emerging.

Modern approaches seek to decouple privacy from disparity. For example, the MIU framework proposed by De Min et al. leverages mutual information minimization and calibration to enforce group robustness during forgetting. Similarly, the fair unlearning optimization frameworks designed by Oesterling et al. prove that it is mathematically possible to satisfy statistical privacy guarantees while explicitly preserving parity across demographic groups. The future of the field relies on treating fairness not as an afterthought, but as a core constraint of the unlearning process itself.

Key Foundational Work

To be forgotten or to be fair: unveiling fairness implications of machine unlearning methods (2024) — contributed 3 concept elements to this direction.
CURE4Rec: A Benchmark for Recommendation Unlearning with Deeper Influence (2024) — contributed 3 concept elements to this direction.
Static and Sequential Malicious Attacks in the Context of Selective Forgetting (2023) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_unlearning_induced_bias_66c74873,
  title = {unlearning-induced bias},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unlearning-induced-bias/rdr_66c74873cecf446fa45783f36553b4d3},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_unlearning_induced_bias_66c74873_v1,
  title = {unlearning-induced bias},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unlearning-induced-bias/rdr_66c74873cecf446fa45783f36553b4d3/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake