Abstract
Video Super-Resolution (VSR) aims to reconstruct high-resolution (HR) frames from low-resolution (LR) sequences. While recent frequency-based methods have shown promise in recovering fine textures, they often ignore the directional nature of motion and suffer from spatial information loss. We propose the Directional Frequency Video Super-Resolution (DFVSR) network, which decomposes frames into directional frequency representations. By utilizing an asymmetric U-shaped alignment network and a specialized Directional Frequency-Enhanced Alignment (DFEA) module, DFVSR achieves state-of-the-art performance with superior parameter efficiency compared to leading models like BasicVSR++.
Introduction
The primary goal of VSR is the restoration of high-fidelity details from degraded video sequences. Despite significant progress, current architectures face three main bottlenecks: the treatment of frequency components as static appearance descriptors regardless of motion direction, the inability to distinguish between informative and noisy frequency regions during alignment, and the loss of critical spatial details during the encoding phase. DFVSR addresses these issues by explicitly modeling motion as a combination of directional frequencies and employing an architecture designed for robust spatial recovery.
Methods
The DFVSR framework is built upon several key innovations designed to optimize feature propagation and alignment.
Directional Frequency Representation (DFR)
Using the Discrete Wavelet Transform (DWT), DFVSR decomposes input frames into four distinct components: a low-frequency base ($l$) and three high-frequency components representing horizontal ($h$), vertical ($v$), and diagonal ($d$) directions. This decomposition allows the network to process motion as an amalgamation of directional frequency changes rather than simple pixel displacement.
Asymmetrical U-shaped Unit (DFUnit)
The core processing block of the network is the DFUnit. As shown in Figure 1, the architecture follows an asymmetrical U-shaped design. The encoder progressively integrates high-frequency alignment features into the low-frequency base. Conversely, the decoder features an iterative refinement branch that maintains communication with the encoder to recover spatial information typically lost during downsampling.
Directional Frequency-Enhanced Alignment (DFEA)
To handle temporal alignment, the authors introduce DFEA. Unlike standard Deformable Convolution (DCN), DFEA employs "double enhancements" to prioritize high-fidelity regions. It leverages learned offsets from one directional component to guide others, ensuring more stable and robust alignment across the sequence.
Experiments and Results
DFVSR was evaluated on standard benchmarks including REDS, Vimeo-90K, Vid4, and UDM10, considering both Bicubic (BI) and Blur Downsampling (BD) degradations.
Quantitative Performance and Efficiency
DFVSR demonstrates remarkable efficiency, utilizing only 7.1M parameters—fewer than BasicVSR++ (7.3M) and significantly less than the transformer-based VSR-T (32.6M). Despite its smaller size, it achieved a PSNR of 32.76 dB on the REDS4 (BI) dataset, outperforming BasicVSR++ (32.39 dB).
Qualitative Analysis
Visual comparisons highlight DFVSR's ability to restore sharp edges and fine structural details, such as window slats and text, which are often blurred or distorted by competing models.
Temporal Consistency
Beyond per-frame accuracy, temporal profile evaluations indicate that DFVSR maintains smoother and more consistent transitions over time compared to models like EDVR or BasicVSR, reducing flickering artifacts in reconstructed videos.
Conclusion
DFVSR introduces a paradigm shift by treating motion through the lens of directional frequency. By combining DWT-based decomposition with an asymmetric alignment network, the model achieves a new state-of-the-art in VSR performance while remaining highly parameter-efficient. Future work may involve optimizing inference latency and extending this directional frequency approach to related tasks like video deblurring and frame interpolation.
Summary updated