Abstract
Real-world stereo image super-resolution has a significant influence on enhancing the performance of computer vision systems. Although existing methods for single-image super-resolution can be applied to improve stereo images, these methods often introduce notable modifications to the inherent disparity, resulting in a loss in the consistency of disparity between the original and the enhanced stereo images. To overcome this limitation, this paper proposes a novel approach that integrates an implicit stereo information discriminator and a hybrid degradation model. This combination ensures effective enhancement while preserving disparity consistency. The proposed method bridges the gap between the complex degradations in the real-world stereo domain and the simpler degradations in the real-world single-image super-resolution domain. Our results demonstrate impressive performance on synthetic and real datasets, enhancing visual perception while maintaining disparity consistency.
Introduction
Stereo Image Super-Resolution (SR) aims to generate high-resolution (HR) image pairs from low-resolution (LR) inputs by exploiting the redundant information shared between the left and right views. While standard Single-Image SR (SISR) techniques have advanced significantly, their direct application to stereo pairs often disrupts "disparity consistency"—the precise geometric relationship used to calculate depth.
Furthermore, real-world stereo images suffer from complex, asymmetric degradations where the left and right lenses may exhibit different blur or noise characteristics. Most existing models are trained on simple bicubic downsampling, making them ill-equipped for these "messy" real-world conditions. When processed by standard models, these images often produce artifacts or lose the geometric alignment necessary for downstream tasks like depth sensing or autonomous driving.
Methodology
The proposed framework, RealSCGLAGAN, introduces a two-pronged solution to bridge the gap between synthetic and real-world stereo SR.
Hybrid Degradation Model
To simulate realistic camera artifacts, the authors developed a three-stage degradation pipeline:
- Stage 1: Applies a sequence of blur, resizing, noise, and JPEG compression.
- Stage 2: Randomly shuffles these operations to create a diverse degradation space. Crucially, it introduces different noise and blur kernels for the left and right images to mimic hardware inconsistencies.
- Stage 3: Uses Sinc filtering to prevent common ringing artifacts.
SCGLANet Architecture
The generator (SIG) utilizes a Stereo Cross Global Learnable Attention Module (SCGLAM). To maintain computational efficiency for high-resolution images, it employs "Bucketing" via Super-Bit Locality Sensitive Hashing, which groups similar features together before calculating correlations.
The training is guided by a Stereo Image Discriminator (SID) containing an Implicit Disparity Extraction Module (IDEM). Unlike standard discriminators that only evaluate image quality, IDEM forces the generator to produce images that maintain the correct geometric relationship (disparity) between the views without adding computational overhead at inference time.
Experiments and Results
The model was trained on 800 pairs from Flickr1024 and 60 pairs from Middlebury. It was evaluated against state-of-the-art (SOTA) methods on synthetic benchmarks (KITTI 2012, KITTI 2015, and a custom Flickr1024RS dataset) and real-world captures (StereoWeb20).
As shown in Figure 3, while many SOTA methods like Real-ESRGAN or SwinIR improve visual sharpness, they often fail to preserve the underlying disparity map, leading to significant errors in depth estimation. RealSCGLAGAN achieves a Mean Absolute Disparity Error (MADE) of 2.20 on the Flickr1024RS dataset, significantly outperforming competitors like DASR (3.01). On the real-world StereoWeb20 dataset, the model achieved the highest Naturalness Image Quality Evaluator (NRQM) scores, demonstrating superior visual perception on actual captures.
Conclusion
RealSCGLAGAN effectively addresses the challenges of real-world stereo image super-resolution by combining a sophisticated hybrid degradation model with a disparity-aware discriminator. By preserving geometric consistency while enhancing visual detail, this method provides a robust foundation for downstream computer vision tasks in autonomous driving and depth sensing.
Summary updated