Cycle Contrastive Adversarial Learning for Unsupervised image Deraining

Chen Zhao, Weiling Cai, ChengWei Hu, Zheng YuanarXiv 2024 · DOI · arXiv

Abstract

Single Image Deraining (SID) is a critical task for enhancing visibility in outdoor vision systems. Traditional deep learning approaches often require paired "rainy/clean" images for training, which are difficult to obtain in real-world settings. This paper introduces CCLGAN, a novel unsupervised framework that utilizes Cycle Contrastive Learning (CCL) and Location Contrastive Learning (LCL) to achieve high-quality deraining without paired data. By integrating a pre-trained CLIP model as a semantic anchor and employing a dual-branch contrastive strategy, CCLGAN preserves high-level semantic meaning and spatial consistency. Evaluations on synthetic and real-world benchmarks show that CCLGAN significantly outperforms state-of-the-art unsupervised methods and rivals supervised models in visual quality and quantitative metrics.

Introduction

Single Image Deraining (SID) aims to recover a clean background from an image obscured by rain. While supervised models have shown success, their reliance on paired datasets limits their applicability to real-world scenarios where such pairs are unavailable. Unsupervised methods like CycleGAN have been proposed but often suffer from "channel pollution," where background details are lost or rain information is incorrectly preserved. CCLGAN addresses these challenges by shifting from pixel-level consistency to feature-based contrastive learning, ensuring the model understands both the content and the spatial structure of the scene.

Methodology

The CCLGAN architecture is built upon two primary components designed to decouple the rain layer from the semantic background:

  1. Cycle Contrastive Learning (CCL): This component ensures semantic consistency across the deraining process. It is divided into two branches: Intra-CCL, which utilizes the semantic latent space of a pre-trained CLIP model to pull the reconstructed image toward clean semantic anchors, and Inter-CCL, which operates in a discriminant latent space to isolate and filter the rain layer.
  2. Location Contrastive Learning (LCL): To maintain spatial integrity, LCL maximizes mutual information between corresponding patches in the input and output images. This prevents the "hallucination" of objects and ensures that background elements remain in their correct coordinates.
The CCLGAN architecture (left) and the conceptual logic of the semantic and discriminant latent spaces (right).

Experimental Evaluation

The model was evaluated on the RainCityscapes (synthetic) and SPA (real-world) datasets. Quantitatively, CCLGAN achieved a PSNR of 29.17 dB on RainCityscapes, a significant improvement over the previous unsupervised state-of-the-art, ANLCL (27.42 dB). On the SPA dataset, CCLGAN's performance was found to be comparable to older supervised models, demonstrating its robustness in real-world conditions.

Qualitative results illustrate that CCLGAN produces sharper images with fewer artifacts compared to other unsupervised methods like CUT and DerainCycleGAN. The model effectively removes both heavy rain streaks and the "veiling" haze often associated with precipitation, as shown in the comparison below.

Qualitative comparison of CCLGAN against state-of-the-art methods on a sample rainy scene.

Conclusion

CCLGAN represents a significant step forward in unsupervised image deraining. By leveraging CLIP as a semantic teacher and employing localized contrastive losses, the framework overcomes the limitations of traditional pixel-based unsupervised learning. While the model introduces some computational overhead due to the CLIP integration, its ability to generalize to real-world data without paired training sets makes it a powerful tool for outdoor vision applications. Future research will explore extending this universal unsupervised strategy to other adverse weather conditions such as fog or snow.

Summary updated

Abstract

To tackle the difficulties in fitting paired real-world data for single image deraining (SID), recent unsupervised methods have achieved notable success. However, these methods often struggle to generate high-quality, rain-free images due to a lack of attention to semantic representation and image content, resulting in ineffective separation of content from the rain layer. In this paper, we propose a novel cycle contrastive generative adversarial network for unsupervised SID, called CCLGAN. This framework combines cycle contrastive learning (CCL) and location contrastive learning (LCL). CCL improves image reconstruction and rain-layer removal by bringing similar features closer and pushing dissimilar features apart in both semantic and discriminative spaces. At the same time, LCL preserves content information by constraining mutual information at the same location across different exemplars. CCLGAN shows superior performance, as extensive experiments demonstrate the benefits of CCLGAN and the effectiveness of its components.
BibTeX
@misc{zhao2024cycle,
  author = {Chen Zhao and Weiling Cai and ChengWei Hu and Zheng Yuan},
  title = {{Cycle Contrastive Adversarial Learning for Unsupervised image Deraining}},
  year = {2024},
  eprint = {2407.11750},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2407.11750},
}
Figure