VisAlloc: Allocating Visual Supervision for On-Policy Self-Distillation

Haoran Xu, Tian Chen, Yuhe Wu, Yanbo Zhao, Junchi Liao, Hongyu Wang, Jiaze Li, Wentao Wang, Yanfeng Li, Rui QianarXiv 2026 · arXiv

TL;DR

VisAlloc is a new framework that improves visual reasoning in Multimodal Large Language Models (MLLMs) by intelligently allocating supervision during training. Instead of treating all data equally, it filters for samples where visual detail is critical and reweights response tokens based on their sensitivity to visual evidence, leading to state-of-the-art performance on fine-grained reasoning benchmarks.

What Problem Does This Paper Address?

Multimodal Large Language Models (MLLMs) often suffer from "lazy reasoning"—they might answer a question based on linguistic priors (guessing based on the text) rather than actually looking at the image. To fix this, researchers use On-Policy Self-Distillation (OPSD). In this setup, a "teacher" model gets "privileged" information (like a high-resolution zoom of a specific object) and guides a "student" model that only sees the full, lower-resolution image.

However, existing methods like Vision-OPD have two major flaws:

  1. Sample Inefficiency: They apply the same amount of supervision to every training example, even if the question is so easy that the model doesn't need the privileged zoom to answer it.
  2. Token Dilution: They supervise every token in a response equally. This "uniform" approach dilutes the signal for tokens that actually require visual grounding (like "blue ribbon") with generic linguistic filler (like "The shows a").

VisAlloc addresses these by asking: How should we allocate visual supervision across training samples and response tokens to maximize learning?

Methods

VisAlloc introduces a dual-level optimization framework that moves away from uniform supervision toward a "counterfactual" approach—comparing what the model sees versus what it could see.

Conceptual comparison of OPD, Vision-OPD, and VisAlloc. VisAlloc contrasts positive and negative views to allocate supervision.

1. Sample-Level Visual Filtering

Before training, VisAlloc runs a filter. It creates two views for every image: a positive crop (centered on the relevant evidence) and a negative crop (spatially displaced away from the evidence). It only keeps training samples where a teacher model answers correctly with the positive crop but incorrectly with the negative crop. This ensures the training set is concentrated on examples where the specific visual detail is the deciding factor.

2. Token-Level Counterfactual Reweighting

During training, VisAlloc uses two teachers (EMA copies of the model) to score the student's generated response.

  • View Sensitivity: It calculates the difference in log-probabilities between the teacher seeing the positive crop and the teacher seeing the negative crop. Tokens with a high difference are "view-sensitive"—they clearly depend on the visual evidence.
  • Gating: To avoid over-supervising common phrases (templates), it uses gates that filter out frequent bigrams and tokens where the student is already very close to the teacher.
  • Inverse-Frequency Redistribution: To prevent frequent but less informative tokens from hogging the supervision "budget," VisAlloc redistributes weight toward rarer, more informative tokens within each batch.
Figure 2: Overview of VisAlloc. (A) Paired-view construction produces an evidence-centered positive crop and a spatially displaced negative crop. A fixed paired-teacher filter then retains samples that the positive teacher answers correctly and the negative teacher answers incorrectly, after annotation-quality checks. Each retained sample keeps its full image and paired teacher views. (B) Both teachers score the same student-generated response under aligned prefixes. (C) Realized-token sensitivity supplies the continuous base; sensitivity or teacher-distribution disagreement admits a bonus under shared context gates, and inverse-frequency redistribution preserves its total before mean normalization. (D) Only the positive teacher supplies the weighted JSD target, and EMA updates once after the actor update. The displayed redistribution is for E > 0, and zero-budget batches retain zero bonus. The views and bonuses are schematic (λ = 4, 4B).

Datasets

The authors trained their models using 6,241 paired visual-evidence examples derived from the Vision-OPD dataset, which were then filtered down to 5,795 high-quality samples.

Evaluation was performed across a comprehensive suite of eight benchmarks:

  • Fine-grained tasks: V*Bench, ZoomBench, HR-Bench 4K, and HR-Bench 8K.
  • Holdout/General tasks: MMVP, CV-Bench, MMStar, and POPE.
  • Additional: MME-RealWorld (EN/CN) for broader context.

Key Insights and Ideas

  • Contrast is More Informative Than Zoom: Simply showing a model a zoomed-in image (Vision-OPD) tells it what is there. Showing it both a zoomed-in image and a "wrong" crop (VisAlloc) tells it why that specific information matters.
  • The "Budget" Perspective: Instead of just adding more supervision, VisAlloc treats supervision as a finite budget. By redistributing this budget away from "easy" samples and "generic" tokens, the model learns more efficiently.
  • Geometry-Matched Crops: To ensure the difference between teachers is purely about the content of the crop and not the shape or resolution, VisAlloc ensures both positive and negative crops share the same rendering geometry.

Key Results

VisAlloc consistently outperforms previous on-policy distillation methods across different model scales (4B and 9B parameters).

  • 4B Model: Achieved a macro-average of 81.44% across eight benchmarks, outperforming the Vision-OPD reproduction by 0.97 points.
  • 9B Model: Achieved a macro-average of 84.92%, a 0.52 point improvement over the baseline.
  • Radar Analysis: As shown in the radar chart below, VisAlloc leads on 8 out of 10 evaluated axes compared to Vision-OPD, showing particularly strong gains in fine-grained benchmarks like V*Bench and ZoomBench.
Radar chart comparing VisAlloc and Vision-OPD across ten benchmarks. VisAlloc leads on eight axes.

Limitations and Future Work

  • Teacher Dependency: The sample filter relies on the teacher's correctness. If the teacher model itself has flaws or biases, those will be inherited by the filter.
  • Negative Crop Ambiguity: In some complex images, a "displaced" negative crop might still accidentally contain relevant evidence or make the question ambiguous, potentially sending a confusing signal.
  • Backbone Specificity: The current results are focused on the Qwen3.5 family. While the logic is general, the specific hyperparameters (like the "bonus strength" $\lambda$) might need tuning for other architectures.
  • Future Direction: The authors suggest exploring how this "allocation" logic could be applied to other modalities, such as temporal grounding in video.

Summary updated

Abstract

On-policy self-distillation (OPSD) offers a promising approach to improving visual reasoning in multimodal large language models (MLLMs). However, not all samples benefit equally from privileged visual guidance, and uniform token supervision can dilute visually grounded signals. We propose VisAlloc, a dual-level framework combining sample-level visual filtering with token-level counterfactual reweighting. A fixed paired-teacher filter retains samples answered correctly from an evidence-centered positive crop but incorrectly from a spatially displaced negative crop. At the token level, both EMA teachers score the same student-generated response, and their absolute log-probability differences measure view sensitivity. Additional supervision targets tokens admitted by sensitivity or teacher-distribution disagreement, with shared frozen-prior, template-bigram, and student--teacher divergence gates. Inverse-frequency redistribution preserves the total additional weight within each micro-batch, while the positive teacher remains the sole distillation target. Inference requires only the full-image student. Across eight benchmarks, our 4B and 9B models achieve macro-averages of 81.44 and 84.92, outperforming Vision-OPD reproductions by 0.97 and 0.52 points, respectively. Code on GitHub: https://github.com/Kaelyn01/VisAlloc
BibTeX
@misc{xu2026visalloc,
  author = {Haoran Xu and Tian Chen and Yuhe Wu and Yanbo Zhao and Junchi Liao and Hongyu Wang and Jiaze Li and Wentao Wang and Yanfeng Li and Rui Qian},
  title = {{VisAlloc: Allocating Visual Supervision for On-Policy Self-Distillation}},
  year = {2026},
  eprint = {2611.25069},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25069},
}
Figure