Abstract
Single-image de-raining is a challenging task due to the overlapping of rain streaks and background textures. Traditional convolutional neural networks often struggle with long-range dependencies and detail preservation. This paper introduces the Non-locally Enhanced Encoder-Decoder Network (NLEDN), which integrates non-local operations with dense connections and a pooling-index-guided decoder. By capturing global context and maintaining spatial precision, NLEDN effectively removes complex rain patterns while preserving fine-grained details. Experimental results on synthetic and real-world datasets demonstrate that NLEDN significantly outperforms existing state-of-the-art methods.
Introduction
Single-image de-raining is an ill-posed problem where the goal is to recover a clean background from an image obscured by rain. Unlike video-based methods that can leverage temporal consistency, single-image approaches must rely entirely on spatial cues. Previous deep-learning models have faced two primary challenges: limited receptive fields and detail loss. Standard convolutional layers are local by nature, making it difficult to identify and remove long rain streaks that span large portions of an image. Furthermore, many models treat de-raining as a simple mapping, often resulting in blurry outputs where fine textures are smoothed away.
Proposed Method
The NLEDN architecture follows a symmetrical encoder-decoder structure designed to address these limitations through three key innovations:
- Non-locally Enhanced Dense Blocks (NEDB): This core component integrates a non-local operation with dense connections. The non-local operation computes relationships between a pixel and every other pixel in a feature map, allowing the network to "see" entire rain streaks at once. Dense connections ensure that hierarchical information is exploited effectively throughout the block.
- Pooling Indices Guided Decoding: To prevent the blurring common in standard upsampling, the decoder utilizes "pooling indices" recorded during the encoder's max-pooling steps. This acts as a form of spatial memory, mapping features back to their precise original locations to reconstruct sharp edges.
- Multi-scale Non-local Enhancement: To manage computational costs, the network uses a grid-based approach for high-resolution layers, performing non-local operations within partitioned regions, while global operations are reserved for lower-resolution feature maps.
Experiments and Results
The model was rigorously evaluated on four major synthetic benchmarks: DDN-Data, DIDMDN-Data, Rain100L, and Rain100H. These datasets cover a wide range of rain densities and orientations. NLEDN consistently outperformed state-of-the-art methods such as DDN, JORDER, and DID-MDN. Quantitatively, NLEDN improved the Peak Signal-to-Noise Ratio (PSNR) by an average of 1.07dB over previous best methods. On the particularly challenging Rain100H dataset, it achieved a PSNR of 30.38, a significant leap over JORDER's 25.21.
Qualitatively, the visual results demonstrate NLEDN's superior ability to remove long, overlapping rain streaks that typically leave "ghostly" artifacts in other models. It also maintains much sharper textures in complex areas like zebra stripes and architectural details.
Conclusion
The Non-locally Enhanced Encoder-Decoder Network effectively addresses the dual challenges of global context and detail preservation in single-image de-raining. While the grid-based non-local operation mitigates computational demands, the model remains more resource-intensive than standard CNNs. Future work may involve extending this architecture to other restoration tasks, such as denoising and dehazing, and addressing complex real-world effects like "rain veils" or mist.
Summary updated