Corrupted but Not Broken: Understanding and Mitigating the Negative Impacts of Corrupted Data in Visual Instruction Tuning

Yunhao Gou, Hansi Yang, Zhili Liu, Kai Chen, Yihan Zeng, Lanqing Hong, Zhenguo Li, John Shanklin, Bo Han, James T. Kwok, Yu Zhang

Abstract

Yunhao Gou, Hansi Yang, Zhili Liu, Kai Chen, Yihan Zeng, Lanqing Hong, Zhenguo Li, Qun Liu, Bo Han, James Kwok, Yu Zhang. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

2025.

Introduction

Visual Instruction Tuning (VIT) is essential for developing Multimodal Large Language Models (MLLMs) capable of following complex instructions based on visual inputs. However, the scalability of VIT is often hampered by the prevalence of "corrupted" data—hallucinations, incorrect OCR, or misaligned responses—found in web-crawled or synthetically generated datasets. While conventional wisdom suggests that such noise fundamentally degrades model performance, this research demonstrates that MLLMs are remarkably resilient. The negative impacts of corruption are localized and, crucially, reversible.

Methods: Leveraging Self-Validation

The authors conducted a systematic investigation using LLaVA-1.5, Qwen-2.5, and LLaMA-3.1 models. They propose a robust training paradigm centered on an emergent "self-validation" capability. Traditional data filtering relies on training loss (Perplexity or PPL), but this metric becomes biased when a model is trained on noisy data.

To overcome this, the researchers introduced Val_PPL. By prompting a corrupted model with a meta-question—"Is the response correct? Answer yes or no."—they measure the model's confidence in identifying incorrect samples. The proposed pipeline involves:

  1. Initial Fine-Tuning: Training a "sacrificial" model on the noisy dataset.
  2. Self-Filtering: Using this model to score and filter the dataset via Val_PPL.
  3. Robust Fine-Tuning: Re-training the model using only the high-quality samples identified in step two.

Key Insights: Localization and Emergence

The study reveals two fundamental properties of MLLMs regarding data corruption:

  1. Parameter Localization: The "bad knowledge" acquired from corrupted data is stored in a tiny fraction (~1.4%) of model parameters. Disabling these specific weights, identified via SNIP (Single-shot Network Pruning) scores, can restore performance from 39.1 to 57.1 (against a clean baseline of ~60).
  2. Emergent Self-Correction: The ability to judge sample correctness (Val_PPL) is an emergent property that manifests in models with 3B parameters or more. Smaller models (e.g., 0.5B) lack the internal consistency required to distinguish clean from corrupted data effectively.
The emergence of self-validation (Val_PPL) as a superior metric for separating clean and corrupted data in models with 3B+ parameters.
  1. Restricted Damage: Corruption is often task-specific. For instance, corrupting a model on VQA tasks does not necessarily leak into its performance on "Conversation" tasks, suggesting the damage is restricted to specific response formats.

Key Results

The Val_PPL strategy significantly outperforms standard noise-reduction techniques like MentorNet or Co-teaching. In experiments with a 50% corruption ratio, the average performance of a standard fine-tuned model dropped to 49.13. However, the Val_PPL robust training paradigm recovered nearly all lost performance, achieving a score of 60.17—comparable to the 61.65 score of a model trained on 100% clean data.

Task performance and precision of the Val_PPL method (Ours) compared to simple fine-tuning across varying corruption rates.

Conclusion and Future Work

This work shifts the perspective on data quality in VIT from "pre-cleaning" to "model-driven recovery." By demonstrating that models can effectively "ignore" misleading data through self-validation, the authors provide a scalable path for training robust MLLMs on noisy, large-scale datasets. Future research may explore real-time weight disabling to prevent the learning of corrupted patterns during the initial training phase.

Summary updated

BibTeX
@misc{gou2025corrupted,
  author = {Yunhao Gou and Hansi Yang and Zhili Liu and Kai Chen and Yihan Zeng and Lanqing Hong and Zhenguo Li and John Shanklin and Bo Han and James T. Kwok and Yu Zhang},
  title = {{Corrupted but Not Broken: Understanding and Mitigating the Negative Impacts of Corrupted Data in Visual Instruction Tuning}},
  year = {2025},
  eprint = {2502.12635},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.12635},
}
Figure