When we fine-tune a large model or apply a machine unlearning algorithm, we expect the targeted information to be permanently deleted from the model's parameters. However, a growing body of research reveals a profound asymmetry: while information introduced during brief fine-tuning stages is easily erased, knowledge baked in during the massive pre-training phase is remarkably resilient. It behaves like an indelible ink, resisting standard gradient updates and frequently resurfacing under adversarial testing.
Understanding why pre-trained knowledge is so stubborn requires looking beneath surface-level model outputs and examining how optimization trajectories interact with deeply embedded parameter spaces.
The Pre-training Weight Anchor
Why does naive fine-tuning or unlearning fail to clear pre-trained facts? The answer lies in the geometry of overparameterized loss landscapes.
When a model is trained on a massive corpus, its weights are configured into a highly stable, deeply reinforced distribution. As shown by Jinhui Xu et al. in Why Fine-Tuning Struggles with Forgetting in Machine Unlearning? Theoretical Insights and a Remedial Approach and further analyzed by Meng Ding et al. in Understanding Fine-tuning in Approximate Unlearning: A Theoretical Perspective, optimization processes in overparameterized regimes naturally converge to solutions that remain mathematically close or "tethered" to the initial weight configuration.
Initial Pre-trained Weights (Strong Anchor)
│
▼
┌─────────────────┐
│ Unlearning/FT │ ──► Low loss achieved on remaining data,
│ Optimization │ but weights remain near the anchor.
└─────────────────┘
│
▼
Residual Pre-trained Features Retained
Because the pre-trained weights act as a powerful anchor, the model can easily find a nearby solution that achieves zero loss on a new or remaining dataset without dismantling the underlying features and residual gradients associated with the data intended for removal. The model satisfies the objective function by taking the path of least resistance, leaving the core foundational representations intact.
The Illusion of Forgetting
Because the underlying representations remain largely unaltered, many unlearning methods merely "hide" information rather than deleting it.
Weitao Ma et al. demonstrated this "fragility gap" in Unveiling Entity-Level Unlearning for Large Language Models: A Comprehensive Analysis, showing that current techniques only mask surface-level associations instead of altering the core parameters where foundational knowledge resides. To prove how superficial this erasure is, Aghyad Deeb et al. developed an adversarial evaluation method called Retraining on T (RTT) in Do Unlearning Methods Remove Information from Language Model Weights?. By retraining an "unlearned" model on a tiny subset of related but independent facts, the authors successfully recovered 88% of the model's original performance. This demonstrated that the pre-trained knowledge was never truly gone; its activation pathways were simply suppressed.
This persistence also distorts downstream reasoning. Even when a model's weights are updated or edited to reflect new information, the original pre-trained signals often remain stronger than the new edits. As a result, when faced with complex inference chains, the model frequently reverts to its deeply embedded pre-trained priors, a phenomenon documented by Zengkui Sun et al. in Outdated Issue Aware Decoding for Factual Knowledge Editing.
The Collateral Damage of Forced Erasure
If pre-trained knowledge is so deeply integrated, what happens if we force the model to forget it using aggressive gradient updates? The results are often catastrophic for the model's general utility.
Kazuki Egashira et al. introduced the PULSE evaluation protocol in PULSE: Practical Evaluation Scenarios for Large Multimodal Model Unlearning to study this exact tension. The authors discovered that attempting to force the removal of core pre-trained concepts—such as real-world identities or foundational facts—severely degrades the model's general reasoning capabilities.
Because pre-trained data forms the structural scaffolding of the model's latent space, tearing out a pre-trained thread causes collateral damage to unrelated capabilities. The model does not just forget the targeted fact; it loses its broader capacity to reason.
Ubiquity Across Modalities and Training Phases
This resilience is not a quirk of text-based language models; it is a fundamental characteristic of large-scale pre-training across modalities and training pipelines:
- Vision-Language Models (VLMs): Massimiliano Mancini et al. showed in The Phantom Menace: Unmasking Privacy Leakages in Vision-Language Models that fine-tuning on strictly anonymized data fails to overwrite private identities learned during pre-training. Similarly, Kodai Kawamura et al. demonstrated in Approximate Domain Unlearning for Vision-Language Models that the difficulty of removing domain knowledge (like real-world imagery) is directly correlated with its prevalence and volume in the original pre-training corpus.
- Video Diffusion Models: In generative video modeling, Chen Chen et al. found in Investigating Memorization in Video Diffusion Models that fine-tuning on specialized or high-quality datasets is insufficient for reliable data unlearning, as the model continues to regurgitate memorized training samples from its initial pre-training phase.
- Alignment Phases: Even downstream alignment phases designed to shape model behavior fail to overwrite this data. Jamie Hayes et al. demonstrated in Measuring memorization in RLHF for code completion that data memorized during early training steps stubbornly persists through Reinforcement Learning from Human Feedback (RLHF), proving that preference alignment does not effectively overwrite sensitive information encoded in the weights.
The Dual Nature of Resilience
While this resistance to erasure poses a severe challenge for privacy, copyright, and safety compliance, it is simultaneously the secret weapon behind a model's stability. The exact same structural resilience that makes pre-trained knowledge hard to unlearn also protects the model from catastrophic forgetting during sequential task learning.
As Thomas Scialom et al. discovered in Fine-tuned Language Models are Continual Learners, massive self-supervised pre-training provides a structural resilience to weight overwriting that is entirely absent in models trained from scratch. Because the pre-trained foundation resists being overwritten, instruction-tuned models can progressively learn new, diverse tasks while maintaining nearly 100% of their original performance using a rehearsal buffer of just 1%.
Ultimately, pre-training knowledge resilience is a double-edged sword. It grants foundation models their robust, unshakeable general intelligence and resistance to forgetting, but it also means that once a piece of information is baked into the core parameters of a model, truly extracting it requires fundamentally rethinking our approach to optimization and weight manipulation.
Key Foundational Work
Fine-tuned Language Models are Continual Learners (2022) — contributed 2 concept elements to this direction.
Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks (2019) — contributed 1 concept elements to this direction.