PromptSR: Cascade Prompting for Lightweight Image Super-Resolution

Wenyang Liu, Chen Cai, Jianjun Gao, Kejun Wu, Yi Wang, Kim-Hui Yap, Lap-Pui Chau

TL;DR

Lightweight Vision Transformers (ViTs) for image super-resolution typically suffer from limited receptive fields due to restrictive window-based attention mechanisms. PromptSR solves this by introducing low-dimensional "anchor prompts" derived from downscaled features, enabling efficient global context aggregation and coarse-to-fine local refinement. The resulting model achieves state-of-the-art performance on standard benchmarks while keeping the parameter count strictly under 1 million.

What Problem Does This Paper Address?

Image super-resolution (SR) reconstructs high-resolution images from low-resolution inputs. Recently, Vision Transformers (ViTs) have dominated this space due to their ability to model long-range pixel dependencies. However, standard self-attention scales quadratically with image size, making it too computationally expensive for resource-constrained mobile or edge devices.

To build lightweight models (typically < 1M parameters), researchers use window-based self-attention, restricting attention to small, localized patches (e.g., $8 \times 8$ or $16 \times 16$). While this saves compute, it severely limits the model's receptive field. Existing workarounds like sliding windows or window-matching strategies still struggle to capture truly global context across the entire image space.

Fig. 1: (a) Prevalent window-based attention strategies, shown at the top row with different colored areas representing windows of adjacent layers, inherently have a limited receptive field at each query position. In contrast, our approach enables global perception via anchor prompts. Cross: cross-attention. (b) PSNR v.s. Model size for Urban100 [16] (×4) between our proposed models with varying numbers of cascade prompting blocks (CPBs) and the latest state-of-the-art lightweight methods. Specifically, we evaluate PromptSR and its enhanced version PromptSR*, to analyze the trade-off between performance and model size.

Methods

PromptSR introduces the Cascade Prompting Block (CPB), a hierarchical architecture that replaces standard Transformer blocks to extract features in a global-to-local and coarse-to-fine manner.

Fig. 1: (a) Prevalent window-based attention strategies, shown at the top row with different colored areas representing windows of adjacent layers, inherently have a limited receptive field at each query position. In contrast, our approach enables global perception via anchor prompts. Cross: cross-attention. (b) PSNR v.s. Model size for Urban100 [16] (×4) between our proposed models with varying numbers of cascade prompting blocks (CPBs) and the latest state-of-the-art lightweight methods. Specifically, we evaluate PromptSR and its enhanced version PromptSR*, to analyze the trade-off between performance and model size.

The CPB consists of three cascaded layers:

  1. Global Anchor Prompting Layer (GAPL): This layer exploits the cross-scale similarity of natural images. It downscales the input feature map to form a set of low-dimensional "Anchors." It then uses cross-attention between these anchors and the full-resolution image tokens to generate Anchor Prompts (APs). Because the APs are low-dimensional, they can interact with the entire image space to capture global context at a fraction of the computational cost ($1/d^2$ complexity reduction, where $d$ is the downscale factor).
  2. Local Prompting Layers (LPLs): The CPB contains two LPLs that refine local details using the global context maps generated by the GAPL.
  3. Coarse Prompting: Uses the coarse similarity map from the GAPL to group similar, non-local tokens into irregularly shaped windows via Category Self-Attention (CSA), breaking the rigidity of standard square windows. This is combined with standard Window Self-Attention (WSA).
  4. Fine Prompting: Repeats this process using a fine-grained similarity map to adaptively refine features down to the pixel level.

Additionally, an Anchor Prompt Update mechanism passes and updates APs across sequential blocks within a Residual Group, ensuring consistent global priors are maintained throughout the network.

Datasets

  • Training: DIV2K (800 images).
  • Evaluation: Five standard public benchmarks: Set5, Set14, BSD100, Urban100, and Manga109.
  • No new benchmarks were introduced.

Key Insights and Ideas

  • Leveraging Cross-Scale Invariance: The core idea relies on the fact that basic structures (lines, edges, textures) repeat across different scales in natural images. Downscaling features to create anchors is an incredibly efficient way to summarize global structures without processing the full-resolution token matrix.
  • Prompting as a Conditional Guide: Instead of forcing the network to learn global dependencies implicitly through stacked local layers, PromptSR explicitly generates global context vectors and feeds them back into local layers as "prompts" to conditionally guide feature reconstruction.
  • Dynamic Windowing via Clustering: By using the GAPL's similarity maps to categorize tokens, the model dynamically clusters related pixels across the image into custom-shaped attention windows, bypassing the spatial limitations of fixed square grids.

Key Results

PromptSR and its enhanced variant, PromptSR*, consistently outperform existing state-of-the-art lightweight models across all datasets:

  • Efficiency vs. Performance: On the challenging Urban100 ($\times4$) dataset, the baseline PromptSR achieves a PSNR of 27.02 dB using only 779K parameters. This outperforms heavy, classical CNN models like RCAN (15.6M parameters, 26.82 dB) and SAN (15.9M parameters, 26.79 dB).
  • Matching Large Transformers: The enhanced PromptSR* (998K parameters) achieves 27.07 dB on Urban100 ($\times4$), matching the performance of the standard SwinIR model, which requires 11.9M parameters—an 11x reduction in model size.
  • Visual Fidelity: Qualitative evaluations show that PromptSR successfully restores sharp, consistent structural details (like fine mesh patterns and text) where other lightweight models produce blurry artifacts or geometric distortions.

Limitations and Future Work

  • Complex Textures: The authors note that the model can struggle with highly intricate, dense, and complex textures (e.g., finely woven networks). In these regions, the coarse-to-fine prompting module can sometimes aggressively over-reconstruct, leading to unwanted artifacts.
  • Loss of Semantic Information: Visualizations using t-SNE revealed that the generated Anchor Prompts do not cluster well by semantic image categories. Because the downscaling process filters out local details to save compute, it inadvertently discards high-level semantic priors that could otherwise guide the super-resolution process. The authors plan to explore semantic-preserving prompting techniques in future work.

Summary updated

Abstract

Although the lightweight Vision Transformer has significantly advanced image super-resolution (SR), it faces the inherent challenge of a limited receptive field due to the window-based self-attention modeling. The quadratic computational complexity relative to window size restricts its ability to use a large window size for expanding the receptive field while maintaining low computational costs. To address this challenge, we propose PromptSR, a novel prompt-empowered lightweight image SR method. The core component is the proposed cascade prompting block (CPB), which enhances global information access and local refinement via three cascaded prompting layers: a global anchor prompting layer (GAPL) and two local prompting layers (LPLs). The GAPL leverages downscaled features as anchors to construct low-dimensional anchor prompts (APs) through cross-scale attention, significantly reducing computational costs. These APs, with enhanced global perception, are then used to provide global prompts, efficiently facilitating long-range token connections. The two LPLs subsequently combine category-based self-attention and window-based self-attention to refine the representation in a coarse-to-fine manner. They leverage attention maps from the GAPL as additional global prompts, enabling them to perceive features globally at different granularities for adaptive local refinement. In this way, the proposed CPB effectively combines global priors and local details, significantly enlarging the receptive field while maintaining the low computational costs of our PromptSR. The experimental results demonstrate the superiority of our method, which outperforms state-of-the-art lightweight SR methods in quantitative, qualitative, and complexity evaluations. Our code will be released at https://github.com/wenyang001/PromptSR.
BibTeX
@article{liu2026promptsr,
  author = {Wenyang Liu and Chen Cai and Jianjun Gao and Kejun Wu and Yi Wang and Kim-Hui Yap and Lap-Pui Chau},
  title = {{PromptSR: Cascade Prompting for Lightweight Image Super-Resolution}},
  journal = {IEEE Trans. Multimedia},
  year = {2026},
  doi = {10.1109/tmm.2026.3651035},
}
Figure