Live-OPSD: On-Policy Self-Distillation for Proactive Streaming Large Video Models

Kaixuan Du, Xin Wan, Dai Guan, Ming Chen, Xiaoxi Jiang, Guanjun JiangarXiv 2026 · arXiv

TL;DR

Live-OPSD is a new post-training method that helps streaming video assistants decide exactly when to speak and what to say. By using "privileged hints" during training to let a model critique its own on-policy generations, it significantly outperforms standard supervised fine-tuning, achieving a 5.26-point gain on the LiveProBench benchmark.

What Problem Does This Paper Address?

Imagine a multimodal AI assistant watching a live video feed of you cooking. If you ask, "Remind me when the water starts boiling," the assistant has two difficult jobs:

  1. Temporal Decision-Making: It must remain silent while the water is cold but speak immediately once bubbles appear.
  2. Content Generation: It must provide a helpful response once it decides to trigger.

Most current models are trained using Supervised Fine-Tuning (SFT) on recorded datasets. However, SFT has a major "off-policy" gap: it only teaches the model how to respond at perfectly annotated moments. In the real world, a model might start speaking too early or stay silent too long. SFT doesn't provide a clear signal on how to correct these specific, self-generated mistakes. Furthermore, because "silence" is much more common than "speaking" in a stream, standard training often leads to models that are either too chatty or too passive.

Methods

The authors introduce Live-OPSD (On-Policy Self-Distillation) to bridge this gap. The core idea is to let the model learn from its own behavior using a "teacher" version of itself that has access to extra information.

Figure 1: Live-OPSD training. The unhinted policy generates a continuation, while the same current model uses a privileged hint to score its prefixes. DAS selects scored decision points for a reverse- KL update. The updated policy then supplies the next rollout and teacher.

The process works in three main steps:

  1. On-Policy Rollout: At a specific point in a video stream, the model (the "student") generates a continuation (either a response or a silence token) based only on the visible video context.
  2. Privileged Teacher Scoring: The same model acts as a teacher. However, the teacher is given a privileged hint—a short text description (generated offline by a larger model) that explains the relevant evidence and the intended behavior (e.g., "The water is boiling now; the model should respond"). This hint allows the teacher to produce a more accurate probability distribution for what the next token should be.
  3. Decision-Aware Selection (DAS): Not all moments are equally useful for training. DAS prioritizes "inconsistent" points where the student's decision (to speak or stay silent) disagrees with the ground truth. This ensures the model spends its "learning budget" on its most critical errors.
Figure 3: Decision-Aware Selection. (a) Sampled decisions are grouped as the two inconsistency directions, consistent responses, or consistent silence. (b) DAS fills a budget of K scored points from an inconsistency quota, then consistent responses and silence. The decision label controls selection, not gradient weighting.

The model is updated using a Reverse KL divergence loss, which encourages the student's output distribution to match the teacher's privileged distribution.

Datasets

The researchers trained their models on a dataset of 3,712 training instances derived from public video datasets, augmented with privileged hints.

For evaluation, they used:

  • LiveProBench: A benchmark specifically designed to test proactive response timing across six tasks, including event sensitivity and anomaly alerting.
  • OVO-Bench: Evaluates broad streaming capabilities like real-time perception and backward tracing.
  • OmniMMI: Focuses on proactive alerting (PA) and interaction.

Key Insights and Ideas

  • Self-Distillation via Context: You don't necessarily need a larger "Teacher" model (like GPT-4) to provide the signal. The same model can teach itself if you simply provide it with better context (the privileged hint) during the training phase.
  • The Power of "Why": Standard SFT tells the model what the answer is. The privileged hints in Live-OPSD tell the model why it should be responding, which helps the model internalize the relationship between visual evidence and the decision to speak.
  • Reverse KL vs. Forward KL: The authors found that Reverse KL divergence is more stable for this task. It forces the model to focus on the specific paths the student is already taking, rather than trying to cover every possible alternative the teacher might suggest.
  • Addressing the Silence Imbalance: By using DAS and specific weighting for silence tokens, the method prevents the model from becoming "lazy" (always staying silent) or "hallucinatory" (speaking when nothing is happening).

Key Results

The authors tested Live-OPSD on two backbones: Qwen2.5-Omni-7B and Qwen3-Omni-30B-A3B.

  • LiveProBench: The 30B model achieved an average score of 59.64, beating the strongest SFT baseline by 5.26 points. It showed particular strength in "Window Response" and "Anomaly Alerting."
  • OmniMMI: On the Proactive Alerting (PA) task, Live-OPSD improved the 7B model's accuracy from 31.50% to 56.50% (+25.0) and the 30B model from 39.50% to 59.00% (+19.5).
  • OVO-Bench: The models remained highly competitive, reaching an overall accuracy of 65.84 on the 30B version, leading in real-time visual perception and backward tracing.
Table 1: LiveProBench results (%). The average is computed from the unrounded Event Sensitivity, Onset Response, Window Response, Anomaly Alert, Sustained Silence, and Duplicate Counting scores. Boldface marks the best result within each block, excluding Live-SFT-Zero.

Limitations and Future Work

  • Offline Hints: Currently, the privileged hints are generated offline by an external model. A truly "self-supervised" version would involve the model generating its own hints or reasoning chains during training.
  • History Mismatch: The model is trained on "recorded" histories (perfectly curated dialogue) but deployed on "model-generated" histories (which might contain its own past errors). Future work could involve multi-turn on-policy rollouts to make the training even more realistic.
  • Action Space: The current model primarily decides between "silence" and "natural language response." Expanding this to include actions like "interrupting the user" or "asking for clarification" would require more complex supervision categories.

Summary updated

Abstract

Streaming multimodal assistants must make response decisions while events are still unfolding. They need to recognize when the available evidence warrants a response, produce an appropriate continuation, and remain silent otherwise. Supervised fine-tuning learns from recorded assistant turns, but does not directly train on the response prefixes produced by the evolving policy itself. We introduce Live-OPSD, an on-policy self-distillation method that uses privileged hints available only during training to supervise the policy on its own outputs. The policy first generates a continuation from the streaming context alone. The same model then re-evaluates that continuation with a hint describing the relevant evidence, and the resulting distributions guide the unhinted policy. On Qwen3-Omni-30B-A3B, Live-OPSD attains an average score of 59.64 on LiveProBench, 5.26 points above the strongest continued-SFT baseline, and stays competitive on OVO-Bench (65.84). These results indicate that guiding the policy on its own outputs with privileged hints improves both when and how a streaming assistant responds.
BibTeX
@misc{du2026live,
  author = {Kaixuan Du and Xin Wan and Dai Guan and Ming Chen and Xiaoxi Jiang and Guanjun Jiang},
  title = {{Live-OPSD: On-Policy Self-Distillation for Proactive Streaming Large Video Models}},
  year = {2026},
  eprint = {2610.36959},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.36959},
}
Figure