Introduction
Single Image Super-Resolution (SISR) has traditionally focused on modest upscaling factors like 2x or 4x. However, "extreme" super-resolution—scaling factors of 10x or greater—presents a massive information gap. In a 16x upscale, a single pixel in a low-resolution (LR) image must be expanded into a 256-pixel (16x16) patch.
Standard feed-forward networks often struggle with this task, producing "fuzzy" textures or losing fine semantic details because they attempt to map the transformation in a single pass. FBRNN (Feedback Recurrent Neural Network) addresses this by treating super-resolution as an iterative refinement process rather than a one-shot estimation.
Architecture and Methodology
The FBRNN architecture is built on the principle of iterative refinement through a feedback loop. Unlike standard deep networks that process data linearly, FBRNN uses a series of connected blocks that process the image over multiple iterations ($T$). This allows the network to refine the image from "coarse to fine" over time.
Core Components
- Recurrent Structure: The model shares weights across iterations, making it highly parameter-efficient.
- Feedback Mechanism: At each step $t$, the high-resolution (SR) output is passed through a Back Projection block. This block projects the SR image back down to the LR space, allowing the network to compare its current estimate with the original input. This acts as a self-correcting loop to identify missing details.
- Sub-network Blocks:
- LR Feature Extraction (LFE): Captures initial features from the input.
- Feedback (FB) Block: Utilizes Residual Dense Blocks (RDB) to exploit hierarchical features.
- Gated Feedback Module (GFM): Manages the flow of information between iterations, deciding what to preserve or update.
- Reconstruction Block (RB): Uses deconvolution layers to upscale features back to the target resolution.
Key Results and Performance
FBRNN was evaluated on high-resolution benchmarks including DIV8K and DIV2K. In the NTIRE 2020 Challenge, the FBRNN-based team (KU_ISPLB) secured 6th place in PSNR and 7th place in SSIM globally.
Quantitative and Qualitative Gains
On the DIV2K validation set at a 16x scale, FBRNN achieved a PSNR of 28.58 dB, significantly outperforming established baselines like D-DBPN (27.16 dB) and SRFBN+ (28.02 dB). As shown in the visual comparison above, FBRNN produces much sharper edges and fewer "halo" artifacts in complex patterns, such as the stained glass window, compared to previous state-of-the-art methods.
Efficiency
One of the most notable aspects of FBRNN is its efficiency. By tying weights across iterations, the 3-iteration version of the model uses only 1.9M parameters. This allows it to outperform much larger models while maintaining a smaller memory footprint.
Conclusion
FBRNN demonstrates that feedback mechanisms and recurrent structures are highly effective for extreme super-resolution tasks. While there remains a trade-off between mathematical fidelity (PSNR) and perceptual "realness" (often achieved by GANs), FBRNN sets a high bar for pixel-perfect reconstruction. Future work may involve integrating GAN-based losses to further enhance the visual texture of the upscaled images.
Summary updated