TL;DR
The Multi-grained Attention Network (MGAN) advances single-image super-resolution by introducing a unified attention mechanism that simultaneously considers feature importance across different scales and spatial regions. By combining multi-scale dense connections with a flexible "multi-grained" attention strategy, the model achieves state-of-the-art reconstruction accuracy with a significantly better performance-to-parameter tradeoff than previous heavyweights like EDSR.
What Problem Does This Paper Address?
Single Image Super-Resolution (SR) is the "ill-posed" challenge of recovering a high-resolution (HR) image from a single low-resolution (LR) input. While Deep Learning has revolutionized this field, two main hurdles remain:
- Uniform Treatment: Standard Convolutional Neural Networks (CNNs) are translation-invariant, meaning they treat every pixel—whether it's part of a flat sky or a complex architectural texture—with equal importance.
- Scale Blindness: Many models ignore the fact that different image features require different receptive fields. A large smooth area needs a wide view, while fine details need a localized focus.
Previous attempts at "attention" (like Squeeze-and-Excitation blocks) focused mostly on which channels were important, often ignoring where in the image those features mattered most. Conversely, spatial attention methods were often computationally expensive.
Methods
The authors propose the Multi-Grained Attention Block (MGAB) as the fundamental building unit of their network. It consists of two primary innovations:
- Multi-scale Dense Connections: Instead of a single convolution path, each block uses parallel 3x3 and 5x5 filters (inspired by Inception) to capture features at different scales. These are then linked via dense skip connections (inspired by DenseNet) to ensure that information from every layer is preserved and gradient flow is maximized during training.
- Multi-grained Attention Mechanism: This is the core novelty. The authors divide feature maps into $S \times S$ grids.
- When $S=1$, the model performs standard Channel Attention (looking at the whole image).
- When $S$ is large (e.g., equal to the image dimensions), it becomes Spatial Attention (looking at individual pixels).
- By using intermediate values (like $S=2$ or $S=4$), the model captures "multi-grained" contextual cues, allowing it to adaptively re-scale feature responses based on surrounding regions.
The final architecture (MGAN) stacks these blocks, uses global feature fusion via a 1x1 bottleneck layer, and finishes with a sub-pixel convolution layer for upscaling.
Datasets
The model was trained on the DIV2K dataset (800 high-quality images). Evaluation was performed on five standard benchmarks:
- Set5, Set14, Bsd100: General image benchmarks.
- Urban100: Focused on urban scenes with many structural details (where MGAN excels).
- Manga109: A dataset of Japanese comics, useful for testing fine-line recovery.
The authors tested against two degradation models: standard Bicubic (BI) downscaling and Blur-downscale (BD).
Key Insights and Ideas
- Attention as a Spectrum: The most significant "aha moment" is that channel attention and spatial attention are not two different things, but two ends of the same "multi-grained" spectrum. By adjusting the grid size $S$, you can unify these strategies into a single framework.
- Structural Superiority: The model performs exceptionally well on the Urban100 and Manga109 datasets. This suggests that the multi-grained approach is particularly effective at recovering repetitive patterns and sharp edges that standard CNNs often "smear."
- Efficiency Matters: The authors demonstrate that you don't need 40+ million parameters (like EDSR) to get top-tier results. By using dense connections and smart attention, MGAN achieves similar or better PSNR (Peak Signal-to-Noise Ratio) with roughly 1/4 the parameters (~11.7M).
Key Results
- Quantitative: MGAN+ (the version using self-ensemble) achieved the highest PSNR/SSIM across almost all tested scales (x2, x3, x4, x8). For example, on Urban100 (x4), MGAN reached 26.47 dB, outperforming the popular MSRN (26.22 dB).
- Qualitative: Visual comparisons show that MGAN recovers straight lines and grid patterns in architectural photos that other models distort or turn into "aliasing" artifacts.
- Model Size: As shown in their comparison plots, MGAN provides a much better "bang for your buck" regarding the number of parameters versus reconstruction accuracy compared to older SOTA models like VDSR or DRCN.
Limitations and Future Work
While the multi-grained approach is powerful, the authors note a trade-off: adding more scales to the attention module increases computational overhead. They eventually settled on $S=1, 2, 4$ as the optimal balance for speed and accuracy.
Future work could involve automatically learning the optimal "grain" for different parts of an image or extending this multi-scale attention to video super-resolution, where temporal consistency adds another layer of complexity.
Summary updated