V-DESIRR: Very Fast Deep Embedded Single Image Reflection Removal

B. H. Pawan Prasad, Green Rosh K. S, R. B. Lokesh, Kaushik Mitra, Sanjoy ChowdhuryICCV 2021 · OpenReview

Abstract

Real-world images often become corrupted due to unwanted reflections, and their removal is highly desirable, especially for high-resolution smartphone photography. Most existing methods either sacrifice processing speed and memory for restoration quality or are limited to low-resolution images. We propose V-DESIRR, a lightweight deep learning model using a novel scale-space architecture. Our method processes images in two stages: a Low Scale Sub-network (LSSNet) for the lowest scale and a Progressive Inference (PI) stage for higher scales. To minimize complexity, PI sub-networks are shallower than LSSNet and utilize weight sharing across scales. This design allows V-DESIRR to generalize to very high resolutions without retraining. Our method is 20× faster with 50× fewer parameters than the state-of-the-art RAGNet, restoring a 12MP image on an Android smartphone in under 5 seconds.

The Challenge of Reflection Removal

Removing reflections from a single image is a classic ill-posed problem. A single pixel contains information from both the background (transmission layer) and the glass reflection. While deep learning has advanced this field, existing models struggle with two main issues:

  1. Computational Cost: Leading models often exceed 100M parameters, requiring massive GPU memory and long processing times.
  2. Resolution Scalability: Models trained on low-res crops often fail on 12MP+ smartphone photos because their receptive fields are too small to capture the global structure of reflections.

Methodology: Scale-Space Architecture

V-DESIRR introduces a hierarchical approach to handle high-resolution data efficiently.

  • LSSNet (Low Scale Sub-network): The process begins at the lowest resolution. A deep U-Net-style network analyzes the entire scene to identify the global context of reflections.
  • Progressive Inference (PI) & HSSNet: As the model moves up the scale pyramid, it uses a shallower High Scale Sub-network (HSSNet) to refine details.
  • Weight Sharing: To maintain a tiny footprint, HSSNet weights are shared across all higher scales. This ensures the model size remains constant regardless of the number of scales used.
  • Convolutional Guided Filters (CGF): These filters upsample results between scales, acting as edge-preserving guides to ensure sharp boundaries in the high-resolution output.
The V-DESIRR Scale-Space Architecture illustrating the flow from LSSNet to Progressive Inference blocks.

Key Insights

The core innovation is the "Shallow Refinement" hypothesis: a deep network is only necessary at the lowest resolution to understand context. Once the context is established, higher resolutions only require shallow refinement. This allows for a dynamic receptive field; by increasing the number of scales ($N$) during inference, the model handles higher resolutions without explicit retraining.

Results and Performance

V-DESIRR sets a new standard for mobile-friendly image restoration. It uses only 2.6 million parameters, a 50x reduction compared to RAGNet's 131 million.

Performance comparison showing V-DESIRR's superiority in speed, memory usage, and complexity vs. PSNR.
  • Speed: It is 20x faster than RAGNet on desktop GPUs and processes 12MP images in under 5 seconds on a Qualcomm Snapdragon 888.
  • Quality: It achieved an average 1.5dB PSNR improvement over RAGNet on standard benchmarks.
  • Efficiency: Peak memory usage is reduced by 3x, making it feasible for devices with limited RAM.

Conclusion

V-DESIRR proves that high-quality reflection removal can be achieved locally on mobile hardware. By prioritizing a scale-space design and weight sharing, the authors have created a model that is not only faster and smaller but also more effective at handling the high resolutions of modern photography. This approach offers a promising template for other mobile image restoration tasks like dehazing and denoising.

Summary updated

Abstract

Real world images often gets corrupted due to unwanted reflections and their removal is highly desirable. A major share of such images originate from smart phone cameras capable of very high resolution captures. Most of the existing methods either focus on restoration quality by compromising on processing speed and memory requirements or, focus on removing reflections at very low resolutions, there by limiting their practical deploy-ability. We propose a light weight deep learning model for reflection removal using a novel scale space architecture. Our method processes the corrupted image in two stages, a Low Scale Sub-network (LSSNet) to process the lowest scale and a Progressive Inference (PI) stage to process all the higher scales. In order to reduce the computational complexity, the sub-networks in PI stage are designed to be much shallower than LSSNet. Moreover, we employ weight sharing between various scales within the PI stage to limit the model size. This also allows our method to generalize to very high resolutions without explicit retraining. Our method is superior both qualitatively and quantitatively compared to the state of the art methods and at the same time 20× faster with 50× less number of parameters compared to the most recent state-of-the-art algorithm RAGNet. We implemented our method on an android smart phone, where a high resolution 12 MP image is restored in under 5 seconds.
BibTeX
@inproceedings{prasad2021v,
  author = {B. H. Pawan Prasad and Green Rosh K. S and R. B. Lokesh and Kaushik Mitra and Sanjoy Chowdhury},
  title = {{V-DESIRR: Very Fast Deep Embedded Single Image Reflection Removal}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  url = {https://openreview.net/forum?id=hbB_vSbP7K},
}
Figure