Corrupted but Not Broken: Understanding and Mitigating the Negative Impacts of Corrupted Data in Visual Instruction Tuning

Yunhao Gou, Hansi Yang, Zhili Liu, Kai Chen, Yihan Zeng, Lanqing Hong, Zhenguo Li, Qun Liu, Bo Han, James T. Kwok, Yu Zhang

TL;DR

Data corruption in visual instruction tuning datasets degrades the performance of Multimodal Large Language Models (MLLMs), but this paper reveals that the damage is highly localized and entirely reversible. By either disabling a tiny fraction (~1.4%) of corruption-affected parameters or leveraging the model's own emergent "self-validation" capability to filter clean data, researchers can effectively restore model performance without relying on expensive human curation or external oracle models.

What Problem Does This Paper Address?

To train Multimodal Large Language Models (MLLMs) to follow visual instructions, researchers rely on Visual Instruction Tuning (VIT). However, scaling up these datasets usually means crawling the web or using generative AI to synthesize data, both of which introduce significant "corruption"—such as hallucinated objects, incorrect answers, or poor OCR quality.

Current methods to mitigate this issue are highly inefficient. They either require expensive human annotators, rely on costly proprietary "oracle" models (like GPT-4) to regenerate data, or use simplistic, rule-based heuristic filters that fail to catch complex errors like logical contradictions or subtle hallucinations. This paper investigates exactly how corrupted data alters an MLLM's internal mechanics and explores whether the models inherently possess the tools to heal themselves.

Methods

The authors systematically analyze data corruption through two main lenses:

  1. Parameter Disabling: Using a network pruning metric called the SNIP score, the authors calculate weight importance on both clean and corrupted datasets. By taking the set difference, they isolate the specific weights responsible for generating corrupted responses. They then simply turn off (zero out) these parameters.
  2. Self-Validation (Val_PPL): Instead of using standard training loss or perplexity (PPL) to identify bad data—which fails when the model has already memorized the corruption—the authors introduce a zero-shot prompting technique. They present the model with a new, unseen evaluation format:
<image>Query: {instruction text}
Response: {response text}
Is the response correct? Answer yes or no:

By measuring the perplexity of the model predicting the word "No" (Val_PPL), the model acts as its own data filter. The highest-scoring clean samples are then used for a brief round of further fine-tuning to fully recover performance.

Figure 1: Examples of corrupted samples in VIT.

Datasets

The models were trained using a 100K instruction-tuning dataset uniformly sampled from LLaVA-665K, which aggregates subsets from various open-source tasks including VQAv2, GQA, OKVQA, OCRVQA, A-OKVQA, LLaVA-158K, ShareGPT, TextCaps, Visual Genome (VG), and RefCOCO.

For evaluation, the authors used 11 standard benchmarks categorized into three formatting groups:

  • VQA: GQA, MME (Perception & Cognition), POPE, OKVQA, TextVQA
  • Multiple-Choice VQA (MC-VQA): MMBench (MMB), SEED-IMG, SciQA-IMG
  • Conversation: LLaVA-Wild, MM-Vet

Key Insights and Ideas

  • MLLMs are Corrupted, Not Broken: The negative effects of bad data are not catastrophic or widespread. The damage is confined to a tiny subset of parameters, meaning the model's core multimodal capabilities remain intact underneath the noise.
  • Task-Isolated Damage: Injecting corruption into one specific response format (e.g., short-phrase VQA) does not degrade the model's performance on tasks utilizing different formats (e.g., long-form conversation), even if they share the same underlying visual knowledge.
  • An Emergent Ability to Self-Correct: The ability to differentiate clean data from corrupted data using Val_PPL is an emergent property observed in models with 3B parameters or more (like Qwen-2.5-3B/7B and LLaMA-3.1-8B), but it is absent in smaller models like Qwen-2.5-0.5B.
  • The Paradox of Better Detection: Counterintuitively, models fine-tuned on more corrupted data actually become better at identifying corrupted samples during self-validation. Training on corrupted data causes a significant distribution shift that sharpens the model's internal awareness of what an incorrect alignment looks like.
Figure 2: Left: Average task performance of MLLMs with various corruption ratios. Though simple fine- tuning suffers from a performance drop, disabling corruption-related parameters (1.4%) can largely restore the performance. Our method is robust to various corrup- tion rates. Right: MLLM’s (fine-tuned with corrupted samples) precisions of classifying clean and corrupted samples. Details are in Appendix A.

Key Results

  • Instant Recovery via Pruning: For a LLaMA-3.1-8B model trained on a heavily corrupted dataset (60% corruption ratio), simply disabling just 1.4% of its parameters restored its average benchmark score from a degraded 39.1 up to 57.1—nearly matching the performance of a model trained purely on clean data (59.3).
  • Outperforming Traditional Baselines: When using the Val_PPL self-validation score to select clean data for further fine-tuning, the corrupted model's average score jumped from 49.13 to 60.17 (under 50% corruption), approaching the pristine baseline of 61.65. This approach significantly outperformed traditional Learning with Noisy Labels (LNL) frameworks like Co-teaching, MentorNet, and JoCoR.

Limitations and Future Work

The primary limitation acknowledged by the authors is scale. Due to computational resource constraints, this study was limited to small and medium-sized MLLMs (up to 8B parameters). The authors did not verify if these exact localized parameter dynamics hold true for frontier-scale models (e.g., 70B or 400B parameters), which remains an open direction for future research.

Summary updated

Abstract

Visual Instruction Tuning (VIT) aims to enhance Multimodal Large Language Models (MLLMs), yet its effectiveness is often compromised by corrupted datasets with issues such as hallucinated content, incorrect responses, and poor OCR quality. Previous approaches to address these challenges have focused on refining datasets through high-quality data collection or rule-based filtering that can be costly or limited in scope. In this paper, we conduct a systematic investigation into the impact of corrupted data on MLLMs and discover that, although corrupted data degrade model performance, such adverse effects are largely reversible, and MLLMs are {\bf corrupted but not broken}. Specifically, we find that disabling a small subset of parameters can almost fully restore performance. Moreover, corrupted MLLMs inherently possess the capability to differentiate between clean and corrupted samples, facilitating dataset cleaning without external intervention. Building on these insights, we introduce a corruption-robust training paradigm that significantly surpasses existing strategies for mitigating the effects of corrupted data.
BibTeX
@misc{gou2025corrupted,
  author = {Yunhao Gou and Hansi Yang and Zhili Liu and Kai Chen and Yihan Zeng and Lanqing Hong and Zhenguo Li and Qun Liu and Bo Han and James T. Kwok and Yu Zhang},
  title = {{Corrupted but Not Broken: Understanding and Mitigating the Negative Impacts of Corrupted Data in Visual Instruction Tuning}},
  year = {2025},
  eprint = {2502.12635},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.12635},
}
Figure