SyncPro: Synchronous Reasoning and Responding for Proactive Streaming Video Understanding

Xueyun Tian, Bingbing Xu, Jiaming Liu, Jinpeng Yu, Yang Shi, Yu Zhao, Ruihua Huang, Yuanzhuo Wang, Huawei ShenarXiv 2026 · arXiv

Imagine a video assistant that doesn't just wait for a video to end to answer your questions, but tracks events in real-time. To do this well, the assistant needs to "think" (track evidence) and "speak" (respond to you). Most current models try to do this in a single, serial stream of text, which often leads to redundant reasoning or delayed responses.

In their latest paper, researchers from Alibaba’s Qwen team and the Chinese Academy of Sciences introduce SyncPro, a framework that allows a video model to reason and respond synchronously using a dual-stream architecture.

TL;DR

SyncPro is a dual-stream framework for proactive streaming video understanding that decouples internal reasoning from user-facing responses. By allowing specialized "Inner" and "Response" branches to activate independently and decode concurrently, it achieves state-of-the-art performance in response timing and accuracy across major proactive benchmarks.

What Problem Does This Paper Address?

Proactive video understanding is the "holy grail" for AI companions—it's the ability to monitor a live stream and alert a user only when a specific condition is met (e.g., "Notify me when the person picks up the yellow cup for the second time").

The authors identify three major bottlenecks in existing single-stream models:

  1. Fixed Schedules: Many models reason at fixed intervals, leading to redundant updates when nothing is happening or delayed responses when events move fast.
  2. Coupled Roles: A single text stream must simultaneously act as a "memory log" for evidence and a "concise assistant" for the user. These goals are often at odds.
  3. Serial Execution: If a model has to finish a long "chain of thought" before it can output a response, the user experiences unnecessary latency.
Requirements for proactive streaming reasoning: (a) Adaptive reasoning, (b) Specialized roles, and (c) Decoupled activation.

Methods

SyncPro solves these issues by introducing a synchronous dual-stream framework built on a shared multimodal backbone (Qwen3.5-4B).

  • Dual Branches: The model uses two specialized sets of Low-Rank Adapters (LoRA). The Inner (I) branch tracks query-relevant evidence, while the Response (R) branch generates user-facing output.
  • Independent Activation: At every "micro-turn" (each new video frame), both branches independently decide whether to stay silent or generate content.
  • Concurrent Decoding: Because the branches are independent, they can decode text in parallel. This significantly reduces the time to the first response token.
  • State Merging: Once decoding finishes, the model merges the KV caches and recurrent states from both branches into a shared history, ensuring that future decisions are informed by both past reasoning and past responses.
The SyncPro architecture featuring independent LoRA routes for Inner and Response branches and a state-merging mechanism.

The model is trained in two stages:

  1. Supervised Fine-Tuning (SFT): Using a new dataset to teach the model how to reason and when to speak. They use a novel Action Temporal Ranking (ATR) loss to sharpen the model's sense of timing.
  2. Trajectory-Level Reinforcement Learning (RL): Using Group Relative Policy Optimization (GRPO) to reward the model for accurate response timing across entire video sequences.

Datasets

The researchers constructed SyncPro-189K, a massive dataset of 189,403 dual-stream trajectories.

  • Construction: They used a mix of "proactive synthesis" (generating complex queries for existing video datasets) and "temporal-label adaptation" (converting standard video annotations into streaming formats).
  • Validation: They used an "Inner-state effectiveness test" to ensure that the internal reasoning generated by the model was actually sufficient to reconstruct the final answer without needing to re-watch the original video frames.

Evaluation was conducted on three major benchmarks: StreamingBench, OVO-Bench, and ProactiveVideoQA.

Key Insights and Ideas

  • Thinking shouldn't block speaking: By allowing the Response branch to trigger based on prior monitoring and current evidence without waiting for current reasoning to finish, SyncPro achieves much lower latency.
  • Adaptive Reasoning is more efficient: Instead of a "fixed budget" of reasoning, SyncPro learns to "think" only when relevant evidence appears. As shown in Figure 4, this avoids redundant updates and "wrong" triggers caused by stale reasoning.
  • Role Specialization works: Swapping the LoRA adapters between branches significantly degrades performance, proving that the model successfully learned distinct "librarian" and "assistant" personas for the two streams.
A comparison showing how adaptive activation avoids the redundant and incorrect updates seen in fixed-schedule reasoning.

Key Results

  • State-of-the-Art Performance: On StreamingBench’s Proactive Output task, SyncPro-RL reached 41.2% accuracy, beating the previous best baseline by 3.2 percentage points.
  • Superior Timing: On OVO-Bench, it achieved a mean F1 score of 34.19%, leading all external baselines.
  • Significant Speedup: Concurrent decoding provided a 1.37x speedup for completing both branches and a 2.2x speedup for generating the first response token compared to serial execution.
  • Preserved Generalization: Despite being tuned for streaming, the model remained within 2 points of the base Qwen3.5-4B on offline benchmarks like MVBench and EgoSchema.

Limitations and Future Work

  • Sampling Rate: The current experiments use a 1-fps sampling rate. While efficient, this might be too slow for extremely fast-paced actions or high-frequency monitoring tasks.
  • KV Cache Management: While the model evicts visual tokens to save memory, the accumulated text tokens from the dual streams still grow over time. Future work could explore more aggressive compression for long-term history.
  • Scaling: The authors focused on a 4B parameter model; scaling this dual-stream approach to larger models (e.g., 72B) could unlock even more sophisticated reasoning capabilities.

Summary updated

Abstract

Proactive streaming video understanding requires assistants to track unfolding events and respond when a user's requested condition is met. To make these decisions, assistants must accumulate relevant evidence and reassess response conditions as new observations arrive. Existing work offers complementary capabilities for this process: proactive video assistants learn when to respond, while streaming video reasoning methods generate scheduled intermediate text to organize observations and support subsequent answers. Together, these capabilities provide a foundation for proactive reasoning, but how to coordinate ongoing reasoning with timely responses remains underexplored. In a direct adaptation, fixed reasoning intervals can trigger updates when nothing relevant has changed or postpone useful updates until the next scheduled step. Sharing one generation path also couples two distinct requirements: preserving evidence for future decisions and producing only the response requested by the user. Moreover, responses generated after reasoning must wait for it to finish, even when sufficient evidence is already available. To address these challenges, we introduce SyncPro, a synchronous dual-stream framework with specialized generation paths for evidence tracking and user-facing responses. Its Inner and Response branches share causal history and activate independently, allowing relevant evidence changes to trigger internal updates and responses to proceed concurrently with ongoing reasoning. We construct 189K dual-stream examples and train SyncPro through supervised fine-tuning followed by trajectory-level reinforcement learning. Experiments across three proactive benchmarks show improved response timing and more accurate answers to user requests.
BibTeX
@misc{tian2026syncpro,
  author = {Xueyun Tian and Bingbing Xu and Jiaming Liu and Jinpeng Yu and Yang Shi and Yu Zhao and Ruihua Huang and Yuanzhuo Wang and Huawei Shen},
  title = {{SyncPro: Synchronous Reasoning and Responding for Proactive Streaming Video Understanding}},
  year = {2026},
  eprint = {2610.32132},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.32132},
}
Figure