QMambaBSR: Burst Image Super-Resolution with Query State Space Model

Xin Di, Long Peng, Peizhe Xia, Wenbo Li, Renjing Pei, Yang Cao, Yang Wang, Zheng-Jun Zha

TL;DR

QMambaBSR introduces a novel framework for burst image super-resolution by adapting State Space Models (Mamba) for multi-frame information retrieval. By utilizing a Query State Space Model (QSSM) to jointly query inter-frame data and scan intra-frame features, the network effectively separates consistent sub-pixel details from random noise. Coupled with an Adaptive Upsampling (AdaUp) module, it achieves state-of-the-art performance, reconstructing sharp, artifact-free high-resolution images from low-resolution bursts.

What Problem Does This Paper Address?

When you take a quick succession of photos on a smartphone (a burst), slight hand tremors cause minor sub-pixel shifts between frames. Burst Image Super-Resolution (BurstSR) exploits these shifts to reconstruct a single, highly detailed, high-resolution (HR) image.

However, two major challenges plague existing BurstSR methods:

  1. Noise vs. Detail Dilemma: Burst images contain high-frequency random noise. Traditional methods fuse frames either by simple concatenation or frame-by-frame processing. This makes it incredibly difficult to distinguish actual sub-pixel details from random noise outliers, often leading to artifacts.
  2. Static Upsampling Blurring: Most architectures rely on static upsampling techniques (like pixel shuffle or fixed interpolations) during the final reconstruction phase. Because these methods apply the same operations regardless of the scene, they fail to adapt to varying sub-pixel distributions, resulting in over-smoothed textures.

Methods

QMambaBSR addresses these bottlenecks through a specialized architecture consisting of three core components:

Figure 2. The overall framework of our proposed QMambaBSR, primarily includes the novel Query State Space Model (QSSM), Multi- scale Fusion Module (MSFM), and the Adaptive Up-sampling Module (AdaUp).

1. Query State Space Model (QSSM)

Standard State Space Models (SSMs) like Mamba excel at processing sequences but lack multi-frame querying mechanisms. The authors modify the discretization process of the SSM so that the base frame (the anchor image) dynamically controls the matrix $B$ and the step size $\Delta$. Meanwhile, the current frames (the remaining burst images) drive the state transitions. This allows the base frame to act as a gate, querying all current frames simultaneously to extract clean sub-pixel data while suppressing random noise in a single step.

Figure 3. Illustrative comparison between the proposed QSSM and existing cross-attention mechanisms.

2. Multi-Scale Fusion Module (MSFM)

To handle details at various scales, MSFM runs three parallel branches:

  • A CNN branch for local feature extraction.
  • An SSM branch with horizontal and vertical scanning for structural patterns.
  • A Channel Transformer branch to capture long-range global context.

3. Adaptive Upsampling Module (AdaUp)

Instead of using fixed upsampling filters, AdaUp uses adaptive pooling and sequence feature interaction to perceive the spatial distribution of sub-pixels in the feature space. It dynamically generates a scene-specific upsampling kernel, ensuring that spatial arrangements are accurately reconstructed without over-smoothing.

Datasets

The authors evaluated their method on four widely accepted BurstSR benchmarks:

  • Synthetic BurstSR: A large-scale synthetic dataset used for baseline training and evaluation.
  • Real BurstSR: Real-world burst images used to test generalization.
  • RealBSR-RAW: A real-world dataset consisting of raw camera sensor data.
  • RealBSR-RGB: A real-world dataset processed into standard RGB color space.

Key Insights and Ideas

  • Exploiting Noise Randomness: The foundational insight of this paper is that while true sub-pixel details maintain a consistent spatial distribution and intensity across burst frames, high-frequency noise appears completely at random (as illustrated in Figure 1). By processing the entire sequence globally rather than frame-by-frame, the network easily flags and eliminates noise outliers.
Visualizing how sub-pixels remain consistent across frames while noise fluctuates randomly.
  • Mamba as a Query Engine: Instead of relying on computationally expensive cross-attention mechanisms to align and query features, QMambaBSR elegantly maps the query-key-value paradigm onto the parameters of a linear State Space Model, drastically improving efficiency while maintaining a progressively diminishing receptive field that favors localized neighborhood interactions.

Key Results

QMambaBSR achieves a new state-of-the-art across both synthetic and real-world datasets:

  • Synthetic BurstSR: Outperforms the previous best method (Burstormer) by 0.29 dB in PSNR, reaching an impressive 43.12 dB.
  • RealBSR-RAW: Surpasses Burstormer by 0.268 dB in PSNR.
  • RealBSR-RGB: Surpasses Burstormer by 0.204 dB in PSNR.

Ablation Study Highlights

An ablation study isolating the core modules demonstrated their individual impact on a baseline network:

  • Adding the MSFM module improved performance by 1.34 dB.
  • Integrating the QSSM added another 0.72 dB.
  • Incorporating the AdaUp module contributed an additional 0.26 dB, validating the choice of dynamic upsampling over static alternatives.

Limitations and Future Work

While QMambaBSR excels at the fusion and upsampling stages of the pipeline, it still relies on external, standard alignment blocks to pre-align the burst frames. The authors note that their future work will focus on:

  1. Utilizing State Space Models to inherently enhance and integrate the alignment stage directly into the burst processing pipeline.
  2. Exploring the architecture's utility in other burst-frame restoration tasks, such as burst denoising and High Dynamic Range (HDR) imaging.

Summary updated

Abstract

Burst super-resolution aims to reconstruct high-resolution images with higher quality and richer details by fusing the sub-pixel information from multiple burst low-resolution frames. In BusrtSR, the key challenge lies in extracting the base frame's content complementary sub-pixel details while simultaneously suppressing high-frequency noise disturbance. Existing methods attempt to extract sub-pixels by modeling inter-frame relationships frame by frame while overlooking the mutual correlations among multi-current frames and neglecting the intra-frame interactions, leading to inaccurate and noisy sub-pixels for base frame super-resolution. Further, existing methods mainly employ static upsampling with fixed parameters to improve spatial resolution for all scenes, failing to perceive the sub-pixel distribution difference across multiple frames and cannot balance the fusion weights of different frames, resulting in over-smoothed details and artifacts. To address these limitations, we introduce a novel Query Mamba Burst Super-Resolution (QMambaBSR) network, which incorporates a Query State Space Model (QSSM) and Adaptive Up-sampling module (AdaUp). Specifically, based on the observation that sub-pixels have consistent spatial distribution while random noise is inconsistently distributed, a novel QSSM is proposed to efficiently extract sub-pixels through inter-frame querying and intra-frame scanning while mitigating noise interference in a single step. Moreover, AdaUp is designed to dynamically adjust the upsampling kernel based on the spatial distribution of multi-frame sub-pixel information in the different burst scenes, thereby facilitating the reconstruction of the spatial arrangement of high-resolution details. Extensive experiments on four popular synthetic and real-world benchmarks demonstrate that our method achieves a new state-of-the-art performance.
BibTeX
@inproceedings{di2025qmambabsr,
  author = {Xin Di and Long Peng and Peizhe Xia and Wenbo Li and Renjing Pei and Yang Cao and Yang Wang and Zheng-Jun Zha},
  title = {{QMambaBSR: Burst Image Super-Resolution with Query State Space Model}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.02149},
}
Figure