Toward Real World Stereo Image Super-Resolution via Hybrid Degradation Model and Discriminator for Implied Stereo Image Information

Yuanbo Zhou, Yuyang Xue, Jiang Bi, Wenlin He, Xinlin Zhang, Jiajun Zhang, Wei Deng, Ruofeng Nie, Junlin Lan, Qinquan Gao, Tong Tong

Abstract

Real-world stereo image super-resolution has a significant influence on enhancing the performance of computer vision systems. Although existing methods for single-image super-resolution can be applied to improve stereo images, these methods often introduce notable modifications to the inherent disparity, resulting in a loss in the consistency of disparity between the original and the enhanced stereo images. To overcome this limitation, this paper proposes a novel approach that integrates an implicit stereo information discriminator and a hybrid degradation model. This combination ensures effective enhancement while preserving disparity consistency. The proposed method bridges the gap between the complex degradations in the real-world stereo domain and the simpler degradations in the real-world single-image super-resolution domain. Our results demonstrate impressive performance on synthetic and real datasets, enhancing visual perception while maintaining disparity consistency.

Introduction

Stereo Image Super-Resolution (SR) aims to generate high-resolution (HR) image pairs from low-resolution (LR) inputs by exploiting the redundant information shared between the left and right views. While standard Single-Image SR (SISR) techniques have advanced significantly, their direct application to stereo pairs often disrupts "disparity consistency"—the precise geometric relationship used to calculate depth.

Stereo image pairs provide redundant information across views but require precise geometric alignment.

Furthermore, real-world stereo images suffer from complex, asymmetric degradations where the left and right lenses may exhibit different blur or noise characteristics. Most existing models are trained on simple bicubic downsampling, making them ill-equipped for these "messy" real-world conditions. When processed by standard models, these images often produce artifacts or lose the geometric alignment necessary for downstream tasks like depth sensing or autonomous driving.

Methodology

The proposed framework, RealSCGLAGAN, introduces a two-pronged solution to bridge the gap between synthetic and real-world stereo SR.

Hybrid Degradation Model

To simulate realistic camera artifacts, the authors developed a three-stage degradation pipeline:

  • Stage 1: Applies a sequence of blur, resizing, noise, and JPEG compression.
  • Stage 2: Randomly shuffles these operations to create a diverse degradation space. Crucially, it introduces different noise and blur kernels for the left and right images to mimic hardware inconsistencies.
  • Stage 3: Uses Sinc filtering to prevent common ringing artifacts.

SCGLANet Architecture

The generator (SIG) utilizes a Stereo Cross Global Learnable Attention Module (SCGLAM). To maintain computational efficiency for high-resolution images, it employs "Bucketing" via Super-Bit Locality Sensitive Hashing, which groups similar features together before calculating correlations.

The training is guided by a Stereo Image Discriminator (SID) containing an Implicit Disparity Extraction Module (IDEM). Unlike standard discriminators that only evaluate image quality, IDEM forces the generator to produce images that maintain the correct geometric relationship (disparity) between the views without adding computational overhead at inference time.

Experiments and Results

The model was trained on 800 pairs from Flickr1024 and 60 pairs from Middlebury. It was evaluated against state-of-the-art (SOTA) methods on synthetic benchmarks (KITTI 2012, KITTI 2015, and a custom Flickr1024RS dataset) and real-world captures (StereoWeb20).

Qualitative comparison of super-resolution results and corresponding disparity maps across various state-of-the-art methods.

As shown in Figure 3, while many SOTA methods like Real-ESRGAN or SwinIR improve visual sharpness, they often fail to preserve the underlying disparity map, leading to significant errors in depth estimation. RealSCGLAGAN achieves a Mean Absolute Disparity Error (MADE) of 2.20 on the Flickr1024RS dataset, significantly outperforming competitors like DASR (3.01). On the real-world StereoWeb20 dataset, the model achieved the highest Naturalness Image Quality Evaluator (NRQM) scores, demonstrating superior visual perception on actual captures.

Conclusion

RealSCGLAGAN effectively addresses the challenges of real-world stereo image super-resolution by combining a sophisticated hybrid degradation model with a disparity-aware discriminator. By preserving geometric consistency while enhancing visual detail, this method provides a robust foundation for downstream computer vision tasks in autonomous driving and depth sensing.

Summary updated

Abstract

Real-world stereo image super-resolution has a significant influence on enhancing the performance of computer vision systems. Although existing methods for single-image super-resolution can be applied to improve stereo images, these methods often introduce notable modifications to the inherent disparity, resulting in a loss in the consistency of disparity between the original and the enhanced stereo images. To overcome this limitation, this paper proposes a novel approach that integrates a implicit stereo information discriminator and a hybrid degradation model. This combination ensures effective enhancement while preserving disparity consistency. The proposed method bridges the gap between the complex degradations in real-world stereo domain and the simpler degradations in real-world single-image super-resolution domain. Our results demonstrate impressive performance on synthetic and real datasets, enhancing visual perception while maintaining disparity consistency. The complete code is available at the following \href{https://github.com/fzuzyb/SCGLANet}{link}.
BibTeX
@misc{zhou2023toward,
  author = {Yuanbo Zhou and Yuyang Xue and Jiang Bi and Wenlin He and Xinlin Zhang and Jiajun Zhang and Wei Deng and Ruofeng Nie and Junlin Lan and Qinquan Gao and Tong Tong},
  title = {{Toward Real World Stereo Image Super-Resolution via Hybrid Degradation Model and Discriminator for Implied Stereo Image Information}},
  year = {2023},
  eprint = {2312.07934},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.07934},
}
Figure