Abstract
Capturing high-quality images in low-light environments is a persistent challenge in computer vision, typically requiring a trade-off between exposure time and motion blur. Traditional methods often address enhancement and deblurring as decoupled tasks, frequently relying on scarce paired datasets. We present VAR-LIDE, a unified, zero-reference framework that leverages pre-trained Visual Autoregressive (VAR) models and Vision-Language Models (VLMs) to simultaneously enhance visibility and remove blur. By utilizing VLM-derived perceptual priors for adaptive modulation, VAR-LIDE achieves state-of-the-art performance on benchmark datasets without requiring ground-truth supervision.
Introduction
Real-world low-light photography is plagued by a "triple threat": insufficient visibility, high sensor noise, and motion blur. Existing solutions often struggle with the "pipeline problem," where sequential enhancement and deblurring steps amplify noise or destroy structural cues. Furthermore, the reliance on paired training data—identical scenes captured in both low-light/blurry and high-light/sharp conditions—limits the scalability and real-world applicability of supervised models.
Methodology
VAR-LIDE employs a pre-trained Visual Autoregressive (VAR) model as its generative backbone. Unlike standard transformers that predict tokens in a 1D sequence, VAR predicts images scale-by-scale (from low to high resolution), naturally preserving structural fidelity. The framework integrates three key components:
- VLM-Informed Conditioning Module (VICM): Uses a VLM to score input visibility, guiding an iterative curve-estimation process to brighten the image optimally for the VAR model without overexposure.
- Spatial-Frequency RoPE (SF-RoPE): A content-aware positional encoding that fuses spatial coordinates with frequency-domain phase information (from FFT) to better resolve structures smeared by blur.
- VLM-Guided Recursive Phase Modulation (VGPM): Refines the image's Fourier phase based on VLM-derived blur scores to eliminate "ghosting" artifacts and sharpen edges.
The system is trained in a zero-reference manner, optimizing internal losses such as exposure control and structural entropy rather than comparing against ground-truth images.
Results and Discussion
The model was evaluated on the LOLBlur and Real-LOLBlur datasets. Quantitatively, VAR-LIDE outperformed traditional cascaded pipelines (e.g., EnlightenGAN + Blur2Blur) by over 5 dB in PSNR on the LOLBlur benchmark. Perceptually, it achieved superior scores in no-reference metrics like NIQE and CLIPIQA, indicating more natural restoration results.
Despite being unsupervised, VAR-LIDE demonstrates performance competitive with fully supervised state-of-the-art models like LEDNet. The inherent ability of pre-trained VAR models to suppress noise, combined with VLM-guided modulation, provides a robust solution for "in the wild" low-light restoration.
Conclusion
VAR-LIDE represents a significant advancement in unsupervised image restoration. By framing joint enhancement and deblurring as a guided generative task, it bypasses the need for paired data while maintaining higher efficiency than diffusion-based methods. Future work will focus on optimizing the architecture for mobile deployment and improving performance in highly cluttered scenes.
Summary updated