DEVIS-GRPO: Unleashing GRPO on Dynamic Extreme View Synthesis

Yi Zuo, Huimin Wu, Lingling Li, Fang Liu, Licheng Jiao, Qing LiarXiv 2026 · arXiv

TL;DR

DEVIS-GRPO is the first online policy gradient framework designed for trajectory-controlled video generation under extreme viewpoints (camera rotations greater than 90°). By decomposing large camera movements into small, manageable increments during training and optimizing the model via reinforcement learning, it achieves unprecedented geometric consistency in a single inference step without requiring expensive, paired extreme-view training data.

What Problem Does This Paper Address?

Controlling a camera's trajectory in AI-generated video is a powerful tool for virtual reality, autonomous driving simulations, and filmmaking. While current video diffusion models handle small camera movements (typically under 40°) reasonably well, they fall apart when asked to execute extreme viewpoint changes, such as a 90° or 120° turn.

Under these extreme conditions, existing models suffer from two major flaws: they either fail to follow the requested camera path entirely, or they generate severe visual inconsistencies, such as drifting foregrounds, warping backgrounds, or intense blurring.

The root cause of this failure is data scarcity. It is incredibly difficult to capture real-world, synchronized video pairs that feature identical dynamic scenes from radically different angles. Without this data, standard supervised learning models cannot generalize to extreme viewpoints.

Extreme viewpoint issues and the ADEVIS solution

Methods

To bypass the lack of extreme-view training data, the authors introduce DEVIS-GRPO, a framework that combines a novel progressive sampling strategy with reinforcement learning.

The pipeline consists of two main pillars:

  1. Accumulative Dynamic Extreme View Synthesis (ADEVIS): Instead of forcing the model to generate an extreme viewpoint all at once during training, ADEVIS breaks the large camera movement down into a sequence of smaller, manageable trajectory increments. The authors experimented with four ways to pass information between these incremental steps, discovering that Bullet Time Accumulation (BTA) worked best. BTA temporarily freezes the scene's action (repeating the frame) while smoothly transitioning the camera angle, creating a geometric bridge that minimizes reprojection errors.
  2. Group Relative Policy Optimization (GRPO): The authors use GRPO—a variant of Reinforcement Learning from Human Feedback (RLHF) that operates without an expensive value model—to fine-tune a baseline video generation model (TrajectoryCrafter enhanced with LoRA layers). During training, the model stochastically samples different trajectory paths to create a "group" of video candidates. These candidates are scored by a custom multi-level consistency-quality reward function that evaluates pixel-level fidelity, semantic similarity (LPIPS), and overall video smoothness. The model is then optimized based on the relative advantages of the samples within the group.

Crucially, while the model uses multi-step accumulation (ADEVIS) to generate diverse training samples and explore the trajectory space, inference is done in a single step. The final optimized model can directly synthesize the extreme-view video in one pass, ensuring high efficiency.

DEVIS-GRPO Pipeline

Datasets

The authors evaluated their method across three distinct datasets to prove its versatility:

  • Kubric-4D: A synthetic dataset used to test extreme-angle camera motions ($|\Delta T| > 120^\circ$) with perfect ground-truth depth maps.
  • iPhone Dataset: A real-world dataset featuring extreme-view video pairs with ground-truth video and depth annotations.
  • DL3DV: A large-scale real-world dataset of static scenes with continuous camera trajectories, adapted by the authors to evaluate dynamic view synthesis by treating the first frame as the source.

Key Insights and Ideas

  • Decouple Training Complexity from Inference Speed: The brilliant insight here is using an expensive, iterative sampling strategy (ADEVIS) purely as a mechanism to bootstrap training data and compute relative rewards. Because the policy model learns the direct mapping, users get the quality of a multi-step geometric alignment at the speed of a single-step generation model.
  • Bullet Time as a Geometric Bridge: Standard 3D reprojection across huge angles leaves massive "empty holes" (occlusions). By repeating frames to create a "bullet time" effect, the model gets a chance to adjust to the new camera perspective before trying to animate the scene further, drastically reducing structural distortion.
  • RL for Hard Geometric Constraints: Traditional supervised fine-tuning struggles to penalize subtle structural drifts. By framing consistency as a reward function in an RL loop, the model quickly learns to prioritize background-foreground alignment.

Key Results

DEVIS-GRPO significantly outperformed state-of-the-art baselines (including GCD, EX-4D, and TrajectoryCrafter) in both quantitative metrics and visual quality:

  • Kubric-4D: Achieved a 21.57% relative improvement in PSNR-Nocc (Peak Signal-to-Noise Ratio in non-occluded areas) and a 7.31% improvement in SSIM-Nocc over the second-best method.
  • iPhone Dataset: Reduced perceptual distance (LPIPS) by 18.56%, proving that the model maintains excellent semantic and stylistic consistency even in real-world settings.
  • Qualitative Performance: Visual results show that DEVIS-GRPO accurately adheres to the specified camera path while keeping the foreground objects and background environments perfectly locked in place, avoiding the indoor/outdoor blending errors seen in competing models.

Limitations and Future Work

The authors noted that DEVIS-GRPO scored slightly lower on certain automated Image Quality (IQ) metrics compared to some baselines. This occurs because competing models often prioritize hallucinating sharp, highly realistic textures at the expense of geometric accuracy (i.e., they generate a pretty picture that doesn't actually match the source video or trajectory). DEVIS-GRPO strictly prioritizes physical and geometric faithfulness.

Future work will focus on striking a better balance between this strict geometric consistency and aesthetic texture generation to boost overall visual crispness.

Summary updated

Abstract

Trajectory-controlled video generation has become essential for controllable video generation. While current methods perform well under small-view camera motions, they degrade significantly with large-view motions. Existing solutions for extreme-view synthesis typically require dedicated video pairs, demanding substantial annotation effort. To address these limitations, we propose Dynamic Extreme VIew Synthesis-GRPO (DEVIS-GRPO), a GRPO-based framework for trajectory-controlled video generation, the first online policy gradient method for extreme view video generation. Central to our approach is a novel sampling strategy: Accumulative Dynamic Extreme VIew Synthesis (ADEVIS), which achieves large-view camera motions by progressively accumulating small-view increments. This method delivers two key advantages: 1) enhanced training efficiency, as it eliminates the need to warm-start the policy model by collecting expensive paired large-view videos, and 2) increased sampling diversity, achieved by flexibly varying trajectory configurations. Finally, we designed a multi-level consistency-quality reward function to select high-quality samples for model optimization. Experiments on the Kubric-4D, iPhone, and DL3DV datasets demonstrate our method's superiority. On Kubric-4D, we achieve relative improvements of 21.57% in PSNR and 7.31% in SSIM over the second-best method in non-occlusion areas. On iPhone, LPIPS is reduced by 18.56%.
BibTeX
@misc{zuo2026devis,
  author = {Yi Zuo and Huimin Wu and Lingling Li and Fang Liu and Licheng Jiao and Qing Li},
  title = {{DEVIS-GRPO: Unleashing GRPO on Dynamic Extreme View Synthesis}},
  year = {2026},
  eprint = {2605.16937},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.16937},
}

Loading related content...

Figure