Abstract
Reference-based super-resolution (Ref-SR) leverages high-resolution (HR) textures from a reference image to enhance a low-resolution (LR) input. This task is complicated by the "transformation gap" (differences in scale and rotation) and the "resolution gap" (matching blurry LR features to sharp HR ones). We introduce $C^2$-Matching (Robust Matching Cross transformation and Cross resolution), a framework that employs contrastive learning and teacher-student correlation distillation to overcome these hurdles. $C^2$-Matching achieves state-of-the-art performance across multiple image and video benchmarks, including a new "in-the-wild" web-referenced dataset.
Introduction
Super-resolution (SR) traditionally relies on internal information from a single image or video sequence. Ref-SR offers a more powerful alternative by providing an external HR reference image. However, the effectiveness of Ref-SR depends entirely on accurate correspondence matching: finding exactly which pixel in the reference image matches which pixel in the input. Existing methods often fail when the reference image is transformed (rotated or scaled) or when the resolution difference between the input and reference is too large. $C^2$-Matching addresses these fundamental gaps to provide robust, high-quality texture restoration.
Methodology: Bridging the Gaps
The $C^2$-Matching framework introduces three core components to ensure robust feature correspondence and texture transfer:
- Contrastive Correspondence Network: To handle the transformation gap, the network is trained using contrastive learning on augmented views. This ensures that feature descriptors remain consistent regardless of scale or rotation changes in the reference image.
- Teacher-Student Correlation Distillation: To bridge the resolution gap, a "teacher" network matches two HR images, providing a high-confidence "ground truth" for correspondences. This knowledge is distilled into a "student" network that learns to match LR inputs to HR references, improving accuracy in complex textured regions.
- Dynamic Aggregation Module: Moving beyond rigid patch-based matching, the system utilizes Deformable Convolutions (DCN) to flexibly gather and align textures from the reference image based on learnable offsets.
Datasets and Experimental Setup
The model was evaluated on standard benchmarks including CUFED5, Sun80, Urban100, and Manga109 for images, and REDS and Vid4 for video. To test real-world robustness, the authors introduced the Webly-Referenced SR (WR-SR) dataset. Unlike previous datasets with curated references, WR-SR uses images retrieved via Google Image search, mimicking a realistic scenario where a user searches the web for a high-quality reference to fix a low-quality photo.
Key Results
$C^2$-Matching demonstrates significant quantitative and qualitative improvements:
- Image SR: It outperforms previous state-of-the-art methods like TTSR and MASA-SR by up to 0.7dB in PSNR on the CUFED5 benchmark.
- Video SR: On the REDS dataset, the method provides a 0.3dB boost over standard video super-resolution techniques.
- Visual Quality: In user studies, over 80% of participants preferred $C^2$-Matching results, noting sharper edges and more realistic texture reconstruction compared to competing methods.
Conclusion
$C^2$-Matching effectively bridges the transformation and resolution gaps that have historically limited Ref-SR performance. By combining contrastive learning with correlation distillation, the framework provides a robust solution for both image and video enhancement. While computationally more intensive than flow-based methods, its ability to handle "in-the-wild" reference images marks a significant step toward practical, high-fidelity super-resolution.
Summary updated