Machine unlearning aims to grant models a "right to be forgotten," allowing developers to scrub sensitive, copyrighted, or harmful data from a pre-trained model without the prohibitive cost of retraining from scratch. While unlearning in text-only models is a maturing field, the rise of Multimodal Large Language Models (MLLMs) has introduced a new complexity: knowledge is no longer stored in a single format. It is distributed across a shared latent space where images, text, and audio are inextricably linked.
The Persistence of Cross-Modal Associations
The fundamental challenge in multimodal unlearning is that knowledge is redundant. If you instruct a model to forget a specific character's textual biography, the model may still recognize that character’s face in an image. As Alexey Dontsov and colleagues demonstrate in CLEAR: Character Unlearning in Textual and Visual Modalities, erasing information from one modality does not guarantee its removal from others.
This "cross-modal persistence" occurs because MLLMs rely on a "projector"—a bridge that maps visual features into the same space as textual tokens. Standard unlearning techniques often fail because they only update the language backbone, leaving the visual-to-textual mapping intact. To achieve true erasure, researchers like Yingzi Ma and Bo Li argue in FIUBENCH that we must jointly optimize both the cross-modal projector and the language model backbone. Without this synchronization, the model retains "latent memories" that can be reactivated by non-textual triggers.
Stabilizing the Vision-Language Bridge
When we force a model to forget, we are essentially pushing its weights away from a learned distribution. In multimodal models, this often leads to "representation collapse," where the delicate alignment between vision and language breaks entirely. The model doesn't just forget the target; it loses the ability to process images altogether, often reverting to repetitive refusals for safe queries.
To combat this, the SineProject framework introduces sinusoidal reparameterization for the projector layers. By stabilizing the geometric alignment during the forgetting process, it prevents the mathematical "weight explosion" that typically plagues gradient-ascent-based unlearning. Similarly, He Li and Haoang Chi propose LUMoE in MLUBench, which uses a Mixture-of-Experts approach to isolate unlearning updates into specific adapters. This ensures that the "forgetting" signal doesn't bleed into the model's core reasoning capabilities.
Surgical Removal via Activation Steering
Rather than permanently altering weights—which can be unstable—some researchers propose "test-time" unlearning. Chaomin Ding and the authors of MLLMEraser use activation steering to suppress sensitive information as it is being processed.
The intuition here is that sensitive knowledge occupies specific "subspaces" in the model's hidden states. By inserting irreversible projection matrices—a method called Metamorphosis Representation Projection (MRP) by Chengcan Wu and Zeming Wei—the model can "surgically" bypass harmful knowledge. Crucially, for multimodal models, these steering vectors must incorporate adversarial images. By capturing the visual recall signals that trigger sensitive data, these methods ensure that the model remains "blind" to the concept regardless of whether the prompt is a sentence or a picture.
The Gap Between Refusal and Erasure
A recurring warning across recent literature is that unlearning is often just obfuscation. In Unlearning vs. Obfuscation: Are We Truly Removing Knowledge?, Guangzhi Sun and colleagues show that while a model might refuse to answer a direct question about a "forgotten" fact, it often still possesses the information, which can be extracted via multiple-choice probes or binary classifiers.
This vulnerability is even more pronounced in generative models. The RECALL attack, detailed by Guanlin Li and Tianwei Zhang in Image Can Bring Your Memory Back, proves that "erased" concepts like nudity or copyrighted styles can be recovered by combining a standard text prompt with a specially optimized image prompt. This suggests that current unlearning methods often only break the link between textual triggers and the target data, leaving the underlying visual representation intact and exploitable.
Future Directions: Continuous and Multimodal Scaling
The field is moving toward continual unlearning, where models must handle a stream of privacy requests over time without access to the original training data. Frameworks like $O^3$, introduced by Chongyang Gao and Lixu Wang in On Large Language Model Continual Unlearning, use Orthogonal LoRA to ensure that new unlearning tasks do not interfere with previous ones.
However, as noted by Mohammad Mohammadi Amiri in OFMU, the scalability of these techniques to even more complex modalities—such as video, audio, or robotic control sequences—remains an open frontier. The goal is a model that can not only learn from everything it sees but also truly forget anything it shouldn't have seen in the first place.