TL;DR
Data corruption in visual instruction tuning datasets degrades the performance of Multimodal Large Language Models (MLLMs), but this paper reveals that the damage is highly localized and entirely reversible. By either disabling a tiny fraction (~1.4%) of corruption-affected parameters or leveraging the model's own emergent "self-validation" capability to filter clean data, researchers can effectively restore model performance without relying on expensive human curation or external oracle models.
What Problem Does This Paper Address?
To train Multimodal Large Language Models (MLLMs) to follow visual instructions, researchers rely on Visual Instruction Tuning (VIT). However, scaling up these datasets usually means crawling the web or using generative AI to synthesize data, both of which introduce significant "corruption"—such as hallucinated objects, incorrect answers, or poor OCR quality.
Current methods to mitigate this issue are highly inefficient. They either require expensive human annotators, rely on costly proprietary "oracle" models (like GPT-4) to regenerate data, or use simplistic, rule-based heuristic filters that fail to catch complex errors like logical contradictions or subtle hallucinations. This paper investigates exactly how corrupted data alters an MLLM's internal mechanics and explores whether the models inherently possess the tools to heal themselves.
Methods
The authors systematically analyze data corruption through two main lenses:
- Parameter Disabling: Using a network pruning metric called the SNIP score, the authors calculate weight importance on both clean and corrupted datasets. By taking the set difference, they isolate the specific weights responsible for generating corrupted responses. They then simply turn off (zero out) these parameters.
- Self-Validation (
Val_PPL): Instead of using standard training loss or perplexity (PPL) to identify bad data—which fails when the model has already memorized the corruption—the authors introduce a zero-shot prompting technique. They present the model with a new, unseen evaluation format:
<image>Query: {instruction text}
Response: {response text}
Is the response correct? Answer yes or no:
By measuring the perplexity of the model predicting the word "No" (Val_PPL), the model acts as its own data filter. The highest-scoring clean samples are then used for a brief round of further fine-tuning to fully recover performance.
Datasets
The models were trained using a 100K instruction-tuning dataset uniformly sampled from LLaVA-665K, which aggregates subsets from various open-source tasks including VQAv2, GQA, OKVQA, OCRVQA, A-OKVQA, LLaVA-158K, ShareGPT, TextCaps, Visual Genome (VG), and RefCOCO.
For evaluation, the authors used 11 standard benchmarks categorized into three formatting groups:
- VQA: GQA, MME (Perception & Cognition), POPE, OKVQA, TextVQA
- Multiple-Choice VQA (MC-VQA): MMBench (MMB), SEED-IMG, SciQA-IMG
- Conversation: LLaVA-Wild, MM-Vet
Key Insights and Ideas
- MLLMs are Corrupted, Not Broken: The negative effects of bad data are not catastrophic or widespread. The damage is confined to a tiny subset of parameters, meaning the model's core multimodal capabilities remain intact underneath the noise.
- Task-Isolated Damage: Injecting corruption into one specific response format (e.g., short-phrase VQA) does not degrade the model's performance on tasks utilizing different formats (e.g., long-form conversation), even if they share the same underlying visual knowledge.
- An Emergent Ability to Self-Correct: The ability to differentiate clean data from corrupted data using
Val_PPLis an emergent property observed in models with 3B parameters or more (like Qwen-2.5-3B/7B and LLaMA-3.1-8B), but it is absent in smaller models like Qwen-2.5-0.5B. - The Paradox of Better Detection: Counterintuitively, models fine-tuned on more corrupted data actually become better at identifying corrupted samples during self-validation. Training on corrupted data causes a significant distribution shift that sharpens the model's internal awareness of what an incorrect alignment looks like.
Key Results
- Instant Recovery via Pruning: For a LLaMA-3.1-8B model trained on a heavily corrupted dataset (60% corruption ratio), simply disabling just 1.4% of its parameters restored its average benchmark score from a degraded 39.1 up to 57.1—nearly matching the performance of a model trained purely on clean data (59.3).
- Outperforming Traditional Baselines: When using the
Val_PPLself-validation score to select clean data for further fine-tuning, the corrupted model's average score jumped from 49.13 to 60.17 (under 50% corruption), approaching the pristine baseline of 61.65. This approach significantly outperformed traditional Learning with Noisy Labels (LNL) frameworks like Co-teaching, MentorNet, and JoCoR.
Limitations and Future Work
The primary limitation acknowledged by the authors is scale. Due to computational resource constraints, this study was limited to small and medium-sized MLLMs (up to 8B parameters). The authors did not verify if these exact localized parameter dynamics hold true for frontier-scale models (e.g., 70B or 400B parameters), which remains an open direction for future research.
Summary updated