Behavior Pack Optimization for Video MLLM Post-Training

Zhaolu Kang, Shiyu Liu, Tailong Luo, Wei Zhang, Yingjie He, Lei Wei, Guansu Wang, Liang He, Siheng Wang, Guangyuan Dong, Jiaqi Su, Shuang Chen, Haoyu Ji, Qishi Zhan, Kaiyue ZhouarXiv 2026 · arXiv

TL;DR

Behavior Pack Optimization (BPO) is a new post-training framework that forces video multimodal models to be truly grounded in visual evidence rather than relying on language shortcuts. By optimizing a "pack" of responses across counterfactual views (like shuffled or masked clips) and enforcing cross-view consistency contracts, BPO significantly improves temporal reasoning and honest abstention when evidence is missing.

What Problem Does This Paper Address?

Current Video Multimodal Large Language Models (MLLMs) are surprisingly easy to fool. While they score high on benchmarks, research shows they often ignore the actual video content. If you shuffle the frames or mask the key event, the models frequently provide the same answer, revealing that they rely on "appearance priors" (recognizing objects) and "language priors" (guessing based on the question) rather than temporal reasoning.

The authors trace this to the optimization granularity of post-training. Standard Reinforcement Learning (RL) methods, like GRPO, grade responses to a single video clip in isolation. The model is never explicitly penalized for being inconsistent—for example, giving the same confident answer to both a normal video and one where the key evidence has been deleted.

Methods

BPO shifts the unit of optimization from a single response to a Behavior Pack.

Figure 1: (a) GRPO samples K rollouts from the original clip v0, centers rewards by the sampled- response mean ¯r, and optimizes each response separately. (b) BPO samples one rollout per view from a question-type-conditioned set, scores the pack jointly with cross-view behavior contracts, and centers advantages relative to the v0 anchor rollout.

1. The Behavior Pack

For every question, BPO generates a set of views: the original clip plus several "counterfactual" versions. These interventions are tailored to the question type:

  • For Event/Binding questions: It uses key-mask (hiding the object), non-key-mask (hiding the background), and span-drop (removing the time segment of the event).
  • For Order/Direction questions: It uses reverse (flipping time) and shuffle (randomizing frames).

2. Cross-View Behavior Contracts

Instead of just checking if the answer is "correct," BPO uses a verifier to enforce three behavioral contracts:

  • Stability: The model should give the same answer if irrelevant background info is masked.
  • Sensitivity: The model’s answer or confidence should change if the key evidence is reversed or removed.
  • Honest Abstention: The model should explicitly state "insufficient evidence" if the diagnostic frames are gone.

3. Anchor-Relative Advantage

Standard RL often uses the mean reward of a group to calculate "advantage." However, video decoding is expensive, meaning groups are usually small, leading to noisy updates. BPO introduces an anchor-relative advantage, using the rollout of the original, unintervened clip as a stable reference (the "anchor") for all other views in the pack.

Figure 2: Overview of Behavior Pack Optimization. Stage A installs the structured EST interface with SFT. Stage B routes each question to a question-typed view set, forms a behavior pack across counterfactual views, scores view-specific and cross-view contract rewards, and updates the policy with anchor-relative advantages.

Datasets

The models were trained using a mix of public video sources and evaluated on several rigorous benchmarks:

  • Primary Evaluation: TempCompass (temporal reasoning), MVBench (comprehensive video tasks), and NExT-QA (causal/temporal action).
  • Transfer Evaluation: Video-MME and LongVideoBench to test generalization to longer videos.
  • Training Metadata: The authors used SAM2 for object masking and shot-boundary detectors to create the counterfactual views.

Key Insights and Ideas

  • Granularity Over Coverage: The bottleneck in video QA isn't just having more data; it's the fact that the loss function doesn't care about cross-view consistency. BPO fixes the objective, not just the distribution.
  • The Power of the Anchor: Using the original clip as a per-prompt anchor significantly reduces variance during training compared to batch-wide normalization. As shown in Figure 3, this leads to much more stable training dynamics and avoids the "KL spikes" common in standard RL.
  • Question-Type Matching: Not all interventions are equal. Shuffling frames helps a model learn temporal order but doesn't help it learn object binding. BPO’s targeted approach (Type 1 vs. Type 2 views) is more efficient than a "shotgun" approach of using all interventions for every question.
Figure 3: Training dynamics: anchor-relative B4 is more stable than batch-normalized B3.

Key Results

BPO was tested primarily on Qwen2.5-VL-7B-Instruct and LLaVA-Video-7B, showing consistent improvements across the board:

  • Accuracy: Improved macro accuracy by 4.7 percentage points (pp) and "temporal-hard" accuracy by 7.8 pp over a vanilla GRPO baseline.
  • Abstention: A massive 20.0 pp increase in Abstention F1, meaning the model is far better at knowing when it doesn't know the answer.
  • Efficiency: These gains were achieved within the same compute budget (FLOPs) as the baselines.
  • Generalization: The gains transferred successfully to different backbones and longer-video benchmarks like Video-MME (+3.8 pp).

Limitations and Future Work

While BPO is a significant step toward grounded reasoning, the authors note a few limitations:

  • Intervention Quality: The method relies on the quality of automated interventions (like SAM2 masks). Artifacts in these masks can sometimes lead to the model "cheating" by spotting the edit rather than reasoning about the content.
  • Scope: The current view set focuses on evidence removal and temporal order. Future work could expand this to open-ended generation and even denser, more complex video scenarios.
  • Verifier Reliability: The system depends on a reliable verifier; while their audit showed 93% accuracy, any verifier error can propagate into the policy.

Summary updated

Abstract

Video multimodal large language models (MLLMs) keep climbing video question answering benchmarks, yet shuffling the frames, masking the segment that supports the answer, or occluding the target object barely changes their predictions. The accuracy rests on appearance and language priors, not on the temporal evidence the question asks for. We trace this to the unit of post-training: rewards are computed on a single response to the original clip, so the model is never asked to behave consistently across views. We propose Behavior Pack Optimization (BPO), which replaces the single response with a behavior pack of outputs across counterfactual views chosen by question type, scored jointly. The pack reward asks for stability when the intervention is irrelevant, sensitivity when key evidence is removed, and abstention when no evidence remains. To keep this objective stable at small pack sizes, BPO uses an anchor-relative advantage: the response on the original view serves as a per-prompt reference instead of a group mean over mixed views. On TempCompass, MVBench, and NExT-QA, BPO improves the macro accuracy of Qwen2.5-VL-7B-Instruct by 4.7 pp, the temporal-hard subset by 7.8 pp, and abstention F1 by 20.0 pp over a budget-matched vanilla GRPO baseline from the same SFT checkpoint. The gains transfer to Video-MME, LongVideoBench, and to LLaVA-Video-7B; ablations confirm they follow the view sets, not the rollout count. We hope this pack-level perspective offers a useful starting point for the video MLLM and multimodal post-training community as the field moves toward evidence-grounded video reasoning.
BibTeX
@misc{kang2026behavior,
  author = {Zhaolu Kang and Shiyu Liu and Tailong Luo and Wei Zhang and Yingjie He and Lei Wei and Guansu Wang and Liang He and Siheng Wang and Guangyuan Dong and Jiaqi Su and Shuang Chen and Haoyu Ji and Qishi Zhan and Kaiyue Zhou},
  title = {{Behavior Pack Optimization for Video MLLM Post-Training}},
  year = {2026},
  eprint = {2610.03141},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.03141},
}
Figure