Abstract: Burst super-resolution aims to reconstruct high-resolution images with higher quality and richer details by fusing the sub-pixel information from multiple burst low-resolution frames. In BusrtSR, the key challenge lies in extracting the base frame's content complementary sub-pixel details while simultaneously suppressing high-frequency noise disturbance. Existing methods attempt to extract sub-pixels by modeling inter-frame relationships frame by frame while overlooking the mutual correlations among multi-current frames and neglecting the intra-frame interactions, leading to inaccurate and noisy sub-pixels for base frame super-resolution. Further, existing methods mainly employ static upsampling with fixed parameters to improve spatial resolution for all scenes, failing to perceive the sub-pixel distribution difference across multiple frames and cannot balance the fusion weights of different frames, resulting in over-smoothed details and artifacts. To address these limitations, we introduce a novel Query Mamba Burst Super-Resolution (QMambaBSR) network, which incorporates a Query State Space Model (QSSM) and Adaptive Up-sampling module (AdaUp). Specifically, based on the observation that sub-pixels have consistent spatial distribution while random noise is inconsistently distributed, a novel QSSM is proposed to efficiently extract sub-pixels through inter-frame querying and intra-frame scanning while mitigating noise interference in a single step. Moreover, AdaUp is designed to dynamically adjust the upsampling kernel based on the spatial distribution of multi-frame sub-pixel information in the different burst scenes, thereby facilitating the reconstruction of the spatial arrangement of high-resolution details. Extensive experiments on four popular synthetic and real-world benchmarks demonstrate that our method achieves a new state-of-the-art performance.
Introduction
Smartphone cameras are physically limited by small sensors and lenses. Burst Super-Resolution (BurstSR) attempts to overcome this by taking a sequence of low-resolution (LR) images and fusing them into one high-resolution (HR) masterpiece. The primary challenge in BurstSR is extracting sub-pixel details—tiny shifts in information caused by natural hand tremor—while simultaneously ignoring high-frequency noise.
Most current methods process frames one by one or simply stack them together. This often fails because the network cannot easily distinguish between a real detail and random noise. Furthermore, standard upsampling techniques (like pixel shuffling) tend to produce "over-smoothed" images, losing the very details the burst was meant to capture.
The QMambaBSR Framework
To address these limitations, the authors propose QMambaBSR, a novel architecture that moves away from traditional Transformer-based or simple CNN-based fusion. The architecture relies on three core innovations: the Query State Space Model (QSSM), the Multi-Scale Fusion Module (MSFM), and the Adaptive Upsampling (AdaUp) module.
Query State Space Model (QSSM)
Taking inspiration from the "Mamba" architecture—a type of State Space Model known for linear complexity—the authors developed a querying mechanism. Instead of just scanning pixels, the "base frame" acts as a controller (a gate) that decides which information to pull from the other frames. Since real details are consistent across frames and noise is random, QSSM can effectively filter out noise by looking at the entire sequence simultaneously.
Multi-Scale Fusion Module (MSFM)
To capture details at different sizes, this module uses a three-pronged approach: a CNN for local features, an SSM for horizontal/vertical global features, and a Transformer block for long-range global context.
Adaptive Upsampling (AdaUp)
Rather than using a fixed mathematical formula to resize the image, AdaUp dynamically generates an upsampling kernel based on the specific scene. This allows the model to adjust its sharpness depending on the content, such as distinguishing between a smooth sky and a textured brick wall.
Key Insights and Results
- Consistency is Key: While noise is chaotic and random, sub-pixel information has a consistent spatial distribution across a burst. By treating the base frame as a "query" in a State Space Model, the network can mathematically suppress anything that doesn't match the expected pattern.
- Mamba Efficiency: While Transformers are powerful, their computational cost grows quadratically with sequence length. QMambaBSR shows that State Space Models can achieve better results with higher efficiency, making them ideal for processing multiple frames.
- Quantitative Performance: On the Synthetic BurstSR dataset, QMambaBSR achieved a PSNR of 43.12 dB, a 0.29 dB improvement over the previous best method. On RealBSR-RAW, it surpassed the SOTA by 0.268 dB.
- Qualitative Quality: Visual comparisons show that QMambaBSR recovers much sharper textures (like text or fine architectural details) and leaves fewer aliasing artifacts. In a blind study with 20 volunteers, the model received the highest average score (8.56/10) for visual quality.
Conclusion
QMambaBSR introduces a novel architecture for Burst Image Super-Resolution that leverages the efficiency of State Space Models. By using a "Query" mechanism to distinguish consistent sub-pixel details from random noise and an adaptive upsampling module to prevent blurring, it achieves new state-of-the-art results. Future work will explore using State Space Models to enhance the alignment stage, potentially leading to a more robust end-to-end Mamba architecture.
Summary updated