Abstract
Snow removal is a critical task in computer vision, aimed at locating snow-covered areas and recovering clean background images. Unlike rain, snow presents diverse patterns and opaque degradations that heavily occlude the background, often requiring computationally expensive models for effective restoration. In this paper, we propose LMQFormer, a lightweight yet high-efficiency snow removal network. Our approach integrates a Laplace-VQVAE to generate a coarse mask as prior knowledge, reducing information entropy and computational costs. We further design a Mask Query Transformer (MQFormer) with a novel Duplicated Mask Query Attention (DMQA) mechanism that constrains attention to snowy regions, drastically reducing parameter counts. Experimental results demonstrate that LMQFormer achieves state-of-the-art quality on popular datasets with significantly fewer parameters and the lowest running time among compared methods.
The Challenge of Snow Removal
Snow removal is significantly more complex than rain removal. While rain is often semi-transparent and follows regular patterns, snow is opaque, varies wildly in shape—from tiny flakes to large clumps—and often creates a "veiling effect" (haze) that obscures the background entirely.
Existing state-of-the-art (SOTA) models are typically computationally heavy, often requiring hundreds of millions of parameters to achieve acceptable results. This makes them impractical for real-world deployment on edge devices, such as security cameras or autonomous vehicle sensors. Furthermore, many models leave "repairing traces"—visible artifacts where the snow used to be. LMQFormer aims to bridge the gap between high-quality restoration and lightweight, real-time performance.
Methodology: LMQFormer Architecture
LMQFormer consists of three primary innovations designed for efficiency:
- Laplace-VQVAE: This module generates a "coarse mask" of snowy areas. Instead of relying on ground-truth masks (unavailable for real-world images), it uses a Laplace operator to highlight high-frequency edges. A Vector Quantized Variational AutoEncoder (VQVAE) then processes this to create a low-entropy prior that guides the recovery network.
- MQFormer (Mask Query Transformer): The main recovery engine uses a dual-encoder architecture. One branch employs transformers to capture global context, while the other uses convolutions for local detail extraction.
- DMQA (Duplicated Mask Query Attention): Standard transformers calculate attention across the entire image, which is computationally expensive. DMQA uses the coarse mask to generate a specific number of "queries" that constrain the transformer's attention specifically to snowy regions. This drastically reduces the parameter count while maintaining high precision.
Key Insights
- Entropy Reduction: By using the Laplace operator to isolate snow edges early, the network reduces the complexity of the recovery task, allowing for a much smaller model footprint.
- Masks as Constraints: In LMQFormer, the mask is not just an input layer but actively shapes the attention mechanism. By forcing queries to align with the mask, the network avoids wasting parameters on clean parts of the image.
- Hybrid Architecture: Combining Transformers for global relationships and Convolutions for local features at low scales allows the model to be efficient without losing the fine details needed for background reconstruction.
Experimental Performance
LMQFormer sets a new benchmark for efficiency in snow removal, as shown in the comparison below.
- Efficiency: LMQFormer uses only 2.18M parameters. For comparison, the previous SOTA model, DDMSNet, uses 229.45M—making LMQFormer roughly 100 times smaller.
- Speed: It achieves a runtime of 0.042 seconds per image, the fastest among all compared SOTA methods (TKL, the next fastest, takes 0.051s but has 13x more parameters).
- Quality: Despite its size, it outperformed larger models in PSNR and SSIM. On the Snow100K dataset, it achieved a PSNR of 31.88 dB, surpassing the next best model (TKL) at 30.51 dB.
- Visual Fidelity: Qualitative results show that LMQFormer is particularly effective at avoiding artifacts on bright red objects (like traffic signs), a common failure point for other desnowing algorithms.
Conclusion
LMQFormer demonstrates that high-quality snow removal does not require massive parameter counts. By leveraging Laplace-guided priors and mask-constrained attention, it achieves real-time performance suitable for edge deployment. Future work will explore extending this architecture to video for temporal consistency in applications like autonomous driving.
Summary updated