Abstract
Video inpainting typically requires precise per-frame masks, a labor-intensive process that limits its accessibility. CIRI (Curricular Inactivation for Residue-aware one-shot video Inpainting) addresses this bottleneck by enabling high-quality object removal using only a single annotated frame. It introduces a feature-level "target inactivation" strategy that is more error-tolerant than traditional image-space masking, alongside an online residue removal process that iteratively cleans up artifacts during inference. Experimental results demonstrate that CIRI significantly improves inpainting quality and temporal consistency across standard benchmarks.
Introduction
Video inpainting—the process of seamlessly filling in missing regions or removing objects—is a staple of modern post-production. However, state-of-the-art models often require a precise mask for every frame, which is a Herculean task for high-frame-rate videos. While "one-shot" methods exist, they frequently fail because slight inaccuracies in mask propagation lead to "residues" (ghostly remnants of the object) or the destruction of useful background information. These errors compound over time, resulting in flickering and blurry artifacts.
Methodology
CIRI is a "plug-and-play" framework designed to enhance existing inpainting models. It shifts the masking process from the image space to the latent feature space through two primary innovations:
1. Curricular Target Inactivation
Instead of using a hard mask to zero out pixels in the image space, CIRI "inactivates" multi-scale feature responses in the latent space. This approach is inherently more error-tolerant because high-level features have larger receptive fields; even if a mask is slightly inaccurate, the network can still leverage surrounding context to understand what needs to be filled. To make the model robust to real-world mask propagation errors, the authors employ Dual-Curriculum Learning, training the model in stages that progress from perfect ground-truth masks to noisy, predicted ones.
2. Online Residue Removal
To address inevitable artifacts, CIRI includes a module that runs during the inference phase. It utilizes a modified object tracking network to detect "inpainting residues"—areas that look like the original object but should not be present. It then performs an iterative "detect-and-refine" process, fine-tuning the mask prediction on-the-fly to scrub away these leftover pixels.
Key Insights
- Feature-level Inactivation: Shifting from masking pixels to inactivating features allows the model to be "fuzzy" about boundaries, which is crucial when mask propagation is imperfect.
- Inference-time Intelligence: By treating inference as an active process with online residue removal, CIRI can correct its own mistakes after the initial pass.
- Universal Compatibility: CIRI acts as a wrapper for various backbones (e.g., STTN, FuseFormer, E2FGVI), consistently boosting their performance in one-shot scenarios.
Results and Evaluation
CIRI demonstrated significant quantitative and qualitative leads over existing one-shot methods. On synthetic datasets, it improved local PSNR (e.g., from 28.11 to 29.01 for the E2FGVI backbone) and significantly reduced VFID scores, indicating smoother, more natural results. Qualitative comparisons show that CIRI effectively removes residues that other methods leave behind, maintaining high temporal coherence.
Conclusion and Limitations
CIRI represents a significant advancement in one-shot video inpainting, making high-quality video editing significantly less laborious. Its primary limitation is the computational overhead introduced by the online residue removal step, which increases inference time. Additionally, the system remains dependent on the quality of the underlying mask propagation; extreme viewpoint changes can still lead to tracking failures. Future work will focus on making mask propagation more robust to such transformations.
Summary updated