Introduction
Super-resolution (SR) is a fundamental task in computer vision, focused on reconstructing high-resolution (HR) images from low-resolution (LR) counterparts. While Single-Image Super-Resolution (SISR) relies on learned image priors, Reference-based Super-Resolution (RefSR) utilizes an additional HR reference image to provide authentic textures. Despite its potential, RefSR faces a significant challenge: the "coupling" of LR and reference features. When these features are processed simultaneously, models often fail to distinguish between relevant and irrelevant information, leading to suboptimal reconstruction.
The Coupling Dilemma: Underuse and Misuse
Existing RefSR methods typically concatenate LR and HR features early in the network. This coupling results in two primary failure modes:
- Reference-underuse: The model produces a blurry compromise between the LR input and the HR reference, failing to fully exploit the available high-frequency details even when the reference is a near-perfect match.
- Reference-misuse: When the reference image is semantically irrelevant to the input, the model may "hallucinate" textures from the reference onto the output, creating unnatural artifacts.
The Task Decoupled Framework
To address these issues, the authors propose a framework that decouples the SR task into three distinct stages:
Stage 1: Structural Super-Resolution
The process begins by upscaling the LR image using a standard SISR backbone (Residual Residual Dense Blocks). This stage focuses exclusively on recovering the global structure and content of the image, ensuring a solid foundation without interference from the reference image.
Stage 2: Dual-Alignment Texture Extraction
To prepare the reference textures, the model employs a hybrid alignment strategy. It uses Flow-based Alignment to correct for global shifts and Deformable Alignment (via DCN) to handle fine-grained local variations. This ensures that the extracted textures are spatially consistent with the upscaled structure from Stage 1.
Stage 3: Adaptive Texture Transfer
The final stage involves "painting" the aligned textures onto the structural base. A critical innovation here is the use of a confidence map. By calculating the similarity between the input and reference features, the model can adaptively decide how much texture to transfer. If the reference is irrelevant, the confidence map suppresses these features, effectively acting as a safety valve against artifacts.
Experimental Results and Impact
The Task Decoupled Framework sets a new state-of-the-art (SOTA) across several benchmarks. On the CUFED5 dataset, it outperformed the previous leading method, $C^2$-Matching, by 0.4dB in PSNR. The gains were even more pronounced on datasets like Urban100 and Manga109, where improvements exceeded 1dB.
Qualitatively, the method produces significantly sharper results. As shown in the figures, it successfully restores intricate details—such as architectural textures and fine text—that competing models either blur or distort. Most importantly, the framework demonstrates remarkable robustness; it maintains high performance even when provided with low-similarity references, where traditional RefSR models often perform worse than simple SISR.
Conclusion
By decoupling structure recovery from texture transfer, this framework overcomes the long-standing issues of reference underuse and misuse in RefSR. While the multi-stage approach and dual-alignment modules introduce some computational complexity, the resulting gains in visual quality and robustness represent a significant advancement for the field. Future work may focus on optimizing the efficiency of these modules and further refining the texture extraction process.
Summary updated