Abstract
DCTMamba is a novel image restoration framework that adapts the high-efficiency Mamba architecture for JPEG deblocking. By integrating the Discrete Cosine Transform (DCT), it establishes a "coarse-to-fine" scanning order that provides the causality Mamba needs, while a scale-adaptive normalization strategy ensures the model remains effective across varying image resolutions.
Introduction
JPEG compression is the backbone of digital imaging, but it comes at a cost: it discards high-frequency information to save space, often resulting in "blocking" artifacts and loss of fine detail. Restoring these images is a classic computer vision challenge. Existing solutions have significant trade-offs: CNNs suffer from a "local reductive bias," while Transformers suffer from quadratic computational complexity. Mamba (State Space Models) offers linear complexity but is designed for causal data, making its application to non-causal 2D images difficult.
Methodology
The DCTMamba architecture bridges the gap between Mamba’s efficiency and the specific nature of JPEG data through three primary components:
Coarse-to-Fine State Space Block (CFSSB)
Instead of scanning pixels left-to-right, the model transforms features into the frequency domain using DCT. It then scans from low-frequency components (the "coarse" structure) to high-frequency components (the "fine" details). This creates a logical, causal sequence: the model first "sketches" the image and then "fills in" the textures.
Scale-Adaptive State Space Block (SASSB)
Frequency distributions change based on image size. SASSB normalizes these coefficients based on the image dimensions, ensuring the model performs consistently regardless of input resolution.
Global-to-Local Block (GLB)
The framework uses a dual-branch approach. One branch uses Mamba for global information reconstruction, while a parallel ResBlock (convolutional) branch handles local feature extraction. The outputs are combined using learned weights.
The model is wrapped in a standard U-Net-style encoder-decoder structure and optimized using a combination of Charbonnier (spatial), DCT (frequency), and SSIM (structural) loss functions.
Key Insights
The most significant innovation is using the frequency spectrum as a proxy for time. By ordering data from low to high frequency, the authors provide a natural "direction" for the state-space model to follow. Additionally, the Scale-Adaptive Normalization makes the model robust for real-world applications where image sizes vary wildly.
Experimental Results
DCTMamba was trained on a massive composite of standard restoration datasets (DIV2K, Flickr2K, BSD500, and WED) and tested across various Quality Factors (QF).
- Quantitative Performance: On the Classic5 dataset (QF10), DCTMamba achieved a PSNR of 30.25 dB, surpassing the previous best of 30.12 dB.
- Qualitative Quality: Visual comparisons show that DCTMamba is significantly better at recovering sharp edges and fine textures (like brickwork or fabric) where other models leave "smudged" artifacts.
- Efficiency: The model achieves better results than the RDN model while using significantly fewer parameters (16.6M vs. 22.12M).
Conclusion
DCTMamba successfully adapts State Space Models for image restoration by leveraging frequency-domain causality. By combining global context via Mamba with local precision via CNNs, it sets a new standard for efficient and effective JPEG deblocking.
Summary updated