Bidirectional Grid Propagation for Video Restoration

This direction is merged into another direction: deformable feature alignment (deformable temporal alignment). Its last version stays readable and citable here.

Version 1 of 115 papers20 conceptsfrozen 2026-10-04

In video restoration tasks—like super-resolution, deblurring, or denoising—the most valuable information for fixing a corrupted frame often resides in other frames. A single frame might be motion-blurred or low-resolution, but the missing details may be sharp and clear just a few milliseconds earlier or later in the sequence.

The core challenge is how to effectively "propagate" this information across time. Recent breakthroughs, led by the development of BasicVSR++ by Kelvin C.K. Chan and colleagues, have shifted the field toward a mechanism known as bidirectional grid propagation. This approach moves away from simple sliding windows or unidirectional streams, instead treating the video as a temporal grid where information is refined through multiple passes.

The Limits of One-Way Thinking

To restore a video, early deep learning models used a "sliding window" approach, looking at a small neighborhood of frames (e.g., frames $t-2$ to $t+2$) to reconstruct frame $t$. However, this ignores long-range dependencies. If a detail is occluded for five frames, a small window will never see it.

Recurrent Neural Networks (RNNs) solved this by passing a "hidden state" forward through the sequence. But unidirectional RNNs have a fundamental flaw: frame $t$ can only benefit from the past. To fix this, researchers like Jiezhang Cao in the Video Super-Resolution Transformer and Sungho Lee in Copy-and-Paste Networks adopted bidirectional propagation. By running one recurrent pass forward and another backward, every frame can finally "see" the entire video's context.

From Bidirectional to Second-Order Grids

While bidirectional propagation is powerful, a single pass in each direction is often insufficient to resolve complex artifacts. BasicVSR++ introduced the concept of second-order grid propagation.

Instead of a simple linear chain, imagine the video sequence as a grid. In the first pass, information flows across the sequence to create an initial estimate. In the second pass, the model "revisits" each frame, but this time it has access to the refined features from the first pass. This iterative refinement allows the network to correct errors made during the initial propagation.

As noted in the NTIRE 2021 Challenge on Video Super-Resolution, this multi-pass approach is significantly more parameter-efficient than simply making a network deeper. It allows the model to aggregate global spatiotemporal information by circulating features back and forth, ensuring that even highly corrupted frames can be recovered using cleaner information from distant parts of the sequence.

Solving the Alignment Problem

Propagation only works if the model knows how to link pixels across frames. If a car moves from the left to the right, the model must "align" the features of the car across time.

The research direction has converged on Flow-Guided Deformable Alignment. Standard optical flow can be brittle, especially with noise or blur. BasicVSR++ uses optical flow as a "guide" to tell a Deformable Convolution (DCN) where to look. The DCN then has the flexibility to sample pixels around that suggested location, making the alignment robust to the very degradations the model is trying to fix. This combination was a key factor in winning multiple tracks of the NTIRE 2021 Challenge on Quality Enhancement of Compressed Video.

Generalization and Real-World Utility

The beauty of this bidirectional grid architecture is its versatility. While originally designed for super-resolution, Kelvin C.K. Chan demonstrated in On the Generalization of BasicVSR++ that the same mechanism outperforms specialized models for deblurring and denoising.

More recently, Haiyu Zhao and the authors of AverNet have applied these principles to "all-in-one" restoration, handling time-varying degradations where a video might shift from being noisy to being blurry. By passing information forward and backward through the sequence, the model can "borrow" clarity from a static, well-lit segment of the video to fix a dark, shaky segment later on.

By treating video restoration as a global, iterative communication problem rather than a frame-by-frame reconstruction task, bidirectional grid propagation has established a new standard for temporal consistency and detail recovery.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_bidirectional_grid_propagation_for_video_6b037ac8,
  title = {Bidirectional Grid Propagation for Video Restoration},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/bidirectional-grid-propagation-for-video-restoration/rdr_6b037ac8fa2c45f9bb68b24b3e8db292},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_bidirectional_grid_propagation_for_video_6b037ac8_v1,
  title = {Bidirectional Grid Propagation for Video Restoration},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/bidirectional-grid-propagation-for-video-restoration/rdr_6b037ac8fa2c45f9bb68b24b3e8db292/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}