Introduction
Learned Video Compression (LVC) has traditionally relied on Motion Estimation and Motion Compensation (MEMC) pipelines. While effective, these methods often depend on explicit "motion signatures" like optical flow, which may not generalize across all video content. Furthermore, standard Convolutional Neural Network (CNN) based models suffer from a limited receptive field, making it difficult to capture global redundancies within a frame.
FLAVC (Feature-Level Attention Video Compression) addresses these challenges by introducing a novel Feature-Level Attention (FLA) mechanism. This approach shifts the focus from traditional motion-heavy estimation toward global feature-domain perception. By combining a Transformer-CNN mixed encoder with dense overlapping patching, FLAVC achieves state-of-the-art rate-distortion performance, significantly outperforming both traditional codecs like HEVC and previous learned models.
Methodology
The FLAVC framework is built upon three primary architectural innovations designed to provide global context and high-fidelity reconstruction without excessive computational overhead.
Feature-Level Attention (FLA)
Instead of the memory-intensive process of calculating attention between every spatial patch, FLA converts local patch embeddings into one-dimensional batch-wise vectors. It then computes a global context matrix across the feature dimension. This allows the model to efficiently perceive redundancies across the entire frame and across time by using previous frames as context.
Dense Overlapping Patcher (DP)
To avoid the edge-detail loss common in traditional non-overlapping block segmentation, the Dense Overlapping Patcher uses a sliding window with overlaps. This creates more detailed, compact patches that preserve local spatial features before they are embedded into the latent space.
Transformer-CNN Mixed Encoder (TCM)
To mitigate the "single-scale bottleneck," the authors developed the TCM encoder. It alternates between Transformer layers for global feature extraction and CNN layers for local feature preservation across four stages of down-sampling. This ensures the resulting latent representation is rich in both detail and context.
Key Results and Performance
FLAVC was evaluated against standard benchmarks including UVG, MCL-JCV, and HEVC Classes B and C. The model achieves significant gains in BD-rate (the bit-rate saved for equivalent quality) compared to the HEVC (HM-16.26) baseline:
- UVG Dataset: -61.46% BD-rate (PSNR) and -72.85% (MS-SSIM).
- MCL-JCV Dataset: -63.76% BD-rate (PSNR).
As shown in the performance plot, FLAVC outperforms the previous leading Transformer-based model, VCT, by a wide margin. The framework is also flexible; the down-scaled FLAVC-S version reduces computational complexity (kMACs/pixel) while still maintaining superior performance over many larger, older LVC models.
Discussion and Limitations
By bypassing explicit motion vectors and identifying temporal redundancies directly in the feature domain, FLAVC reduces reliance on error-prone motion estimation. The synergy between the TCM encoder and the Dense Patcher ensures that global perception does not come at the cost of local detail.
However, some limitations remain. The model's performance curve tends to "stiffen" at lower resolutions, suggesting a potential information bottleneck in the frame feature map at high bit-rates. Additionally, while more efficient than standard Transformers, the full FLAVC model has a relatively high parameter count (143M). Future work will focus on addressing noise sensitivity and performance saturation in very high bit-rate settings.
Summary updated