Introduction
Single Image Super-Resolution (SR) is the task of reconstructing high-resolution (HR) images from their low-resolution (LR) counterparts. While Swin Transformers have set new benchmarks in this field through Window Self-Attention (WSA), they are often limited by a restricted receptive field and high computational demands. This paper introduces the N-Gram context to the vision domain, enabling Swin Transformers to capture broader spatial patterns beyond local windows, and proposes NGswin, an efficient asymmetric U-Net architecture designed for lightweight SR.
Methodology
The authors propose a dual-strategy approach involving a novel contextual mechanism and a specialized network architecture.
N-Gram Context
In Natural Language Processing, an N-Gram refers to a sequence of $N$ words. This paper redefines the N-Gram as a group of neighboring local windows. To maintain efficiency, the model generates "Uni-Gram" embeddings using channel-reducing group convolutions. These embeddings then interact via a Sliding-WSA mechanism, which effectively expands the receptive field. This allows the model to incorporate information from surrounding windows before the standard window partitioning occurs, bridging the gap between isolated local regions.
NGswin Architecture
The NGswin model utilizes an asymmetric U-Net structure to balance performance and efficiency:
- Hierarchical Encoder: A three-stage encoder with patch-merging captures features across multiple scales.
- SCDP Bottleneck: The "Spatial-Channel-Dimensional-Projection" bottleneck aggregates multi-scale outputs from the encoder, providing rich feature representation without the overhead of traditional U-Net bottlenecks.
- Asymmetric Decoder: The decoder is significantly smaller than the encoder, focusing computational resources on the critical early stages of feature extraction.
Key Insights
The core innovation lies in leveraging spatial redundancy. Since images often contain repetitive patterns, a degraded pixel in one window can often be restored by referencing a similar, clearer pattern in a neighboring window. The N-Gram context facilitates this cross-window information flow. Furthermore, the authors demonstrate that the N-Gram context is a plug-and-play module; when applied to existing models like SwinIR-light, it consistently improves performance.
Results and Evaluation
The models were evaluated on standard benchmarks including DIV2K, Set5, Set14, BSD100, Urban100, and Manga109.
Efficiency and Performance
NGswin achieves state-of-the-art results for lightweight SR while drastically reducing computational costs. For a $2\times$ upscaling task, NGswin requires only 140.4G Mult-Adds, a 42% reduction compared to SwinIR-light's 243.7G, while maintaining competitive PSNR.
Qualitative Analysis
Visual results indicate that N-Gram-enhanced models are superior at reconstructing fine, repetitive textures—such as building facades or manga line art—that standard window-based Transformers often blur.
Conclusion
By introducing N-Gram context and the NGswin architecture, this work addresses the fundamental limitations of Swin Transformers in image restoration. The approach provides a significant boost in both receptive field and computational efficiency, making high-quality super-resolution more feasible for resource-constrained environments. Future work may explore extending this context mechanism to other low-level vision tasks like denoising and deblurring.
Summary updated