Beyond the Visible: Disocclusion-Aware Editing via Proxy Dynamic Graphs

Anran Qi, Changjian Li, Adrien Bousseau, Niloy J. MitraarXiv 2025 · DOI · arXiv

Introduction

Modern image-to-video (I2V) models often struggle with "precision control," particularly when animating specific parts of an object or handling disocclusions—the pixels revealed behind a moving object. Current models typically rely on stochastic inpainting to "guess" what lies behind, often leading to temporal inconsistency, "ghosting," or a loss of object identity. This paper introduces a training-free pipeline designed to give users deterministic control over both motion and the content revealed during disocclusion.

Methodology

The proposed framework decouples motion from appearance using a two-pass approach centered on a Proxy Dynamic Graph (PDG).

Overview of the disocclusion-aware editing framework, illustrating the transition from input image to PDG-defined motion and final latent-space compositing.
  1. Motion Definition: Users construct a PDG over the input image. Nodes represent segmented object parts (lifted into 3D via depth estimation), while edges define articulated movements like rotation or translation.
  2. Initial Generation: A "Diffusion-as-Shader" (DaS) model uses the PDG's tracking video to generate a base animation.
  3. User-Guided Reveal: To handle disocclusions precisely, users can edit the final frame of the base video (e.g., using generative fill) to specify exactly what should be revealed.
  4. Latent-Space Compositing: Instead of simple pixel-level pasting, the system blends the latent features of the original image and the user's edit. By injecting these features during the diffusion denoising process, the model harmonizes the new content with the scene's lighting, shadows, and reflections.

Evaluation and Results

To evaluate this new task, the authors established a benchmark of 50 samples focusing on complex articulation and specific "reveals."

A sequence of frames showing the seamless reveal of a user-defined 'bun' as the gingerbread man's hat is lifted, demonstrating high temporal consistency.

The method demonstrates significant improvements over existing state-of-the-art models:

  • Motion Accuracy: Quantitative tests using optical flow showed the proposed method achieving a score of 0.72, vastly outperforming baselines like Veo3.1 (0.02), which rely solely on text prompts.
  • User Preference: In a study with 32 participants, the method was overwhelmingly preferred for its adherence to motion instructions and realism in disoccluded areas compared to models like Puppet-Master and DragAnything.

Key Insights

The core innovation lies in the decoupling of motion and appearance. By using the PDG as a deterministic skeleton and performing compositing in the latent space rather than the pixel space, the model ensures that new content obeys physical laws like shading while maintaining the structural integrity of the moving objects.

Conclusion

While the current system excels at rigid and semi-rigid deformations, future work aims to address highly fluid dynamics (e.g., water or cloth). Nonetheless, this framework provides a predictable, user-centric workflow for high-quality, articulated video generation that goes "beyond the visible" pixels of the source image.

Summary updated

Abstract

Disocclusion occurs when object movement reveals previously hidden content. Existing image-to-video methods provide different forms of motion and disocclusion control, yet the design space they span remains poorly understood. In this work, we systematically explore an algorithmic design space spanning motion specification (text-based versus tracking-based) and disocclusion specification (text-based versus image-based). To facilitate this study, we construct a benchmark consisting of both synthetic scenes and captured scenes, covering rigid and deformable motions, as well as static and dynamic disoccluded objects. We compare existing and adapted methods across this design space to analyze the strengths and limitations of different control strategies for handling disocclusions during motion. Our study reveals that tracking-based guidance improves motion accuracy, while image-based guidance improves the fidelity of newly revealed content. However, jointly controlling the appearance and dynamics of disoccluded objects remains a challenging open problem, particularly in scenarios where newly revealed objects undergo their own motion after becoming visible.
BibTeX
@misc{qi2025beyond,
  author = {Anran Qi and Changjian Li and Adrien Bousseau and Niloy J. Mitra},
  title = {{Beyond the Visible: Disocclusion-Aware Editing via Proxy Dynamic Graphs}},
  year = {2025},
  eprint = {2512.13392},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.13392},
}
Figure