Abstract
Low-light video enhancement (LLVE) requires a delicate balance between visual quality, temporal stability, and computational efficiency. Traditional deep learning models often struggle with real-time constraints, while standard 3D Lookup Tables (LUTs) fail to capture the complex, non-linear color shifts inherent in dark environments. This paper introduces WaveLUT, a novel 4D LUT framework that integrates wavelet-based lighting priors and a multimodal text-driven training strategy. WaveLUT achieves state-of-the-art visual quality and temporal consistency while maintaining the extreme efficiency required for live applications, processing 1080p video at approximately 0.05 seconds per frame.
Introduction
Enhancing video captured in low-light conditions is a significant challenge. The primary goal is to improve visibility without introducing artifacts like flicker or color bias. While deep learning has advanced image quality, the computational overhead often precludes real-time video use. Conversely, Lookup Tables (LUTs) offer high speed but limited capacity; standard 3D LUTs cannot distinguish between different lighting contexts for the same input pixel value, leading to mapping errors.
WaveLUT addresses this "one-to-many" mapping problem by introducing a fourth dimension to the LUT structure, representing enhancement intensity. This allows the model to be context-aware, adapting its color transformation based on the surrounding environment.
Methodology
The WaveLUT architecture is built upon three core innovations designed to optimize color mapping for dynamic, low-light scenes.
1. Wavelet-Prior 4D LUT
Instead of a static 3D (RGB) table, WaveLUT utilizes a 4D structure where the fourth dimension is guided by a lighting prior. The system employs a Discrete Wavelet Transform (DWT) to extract low-frequency components from the input video. These components serve as a stable "rough lighting prior," separating illumination from high-frequency noise and detail.
2. Dynamic Fusion Strategy
Lighting is rarely uniform across a video frame. WaveLUT uses a spatial weighting mechanism to adaptively fuse the wavelet-derived lighting prior with the target intensity structure. This ensures the enhancement is localized and contextually relevant, preventing a "one-size-fits-all" filter effect that often plagues simpler models.
3. Text-Driven Appearance Reconstruction
To ensure perceptual quality, the model is trained using a multimodal approach. It leverages the CLIP (Contrastive Language-Image Pre-training) model to compare enhanced frames against semantic text prompts such as "high light image" and "clear image." This nudges the model toward results that are not just mathematically accurate, but visually pleasing to human observers.
Experimental Results
WaveLUT was evaluated on several prominent benchmarks, including SMID (still videos with long-exposure ground truths) and SDSD (dynamic indoor/outdoor subsets).
Quantitative and Qualitative Performance
- Superior Accuracy: WaveLUT outperformed previous state-of-the-art methods like FASTLLVE, achieving a 0.57 dB PSNR improvement on the SDSD dataset and a 1.07 dB boost on SMID.
- Real-Time Efficiency: As shown in the performance analysis, WaveLUT maintains a low inference time (approx. 0.05s per frame on an RTX 3090), making it suitable for 1080p live video processing.
- Visual Fidelity: Qualitative assessments indicate significantly better color saturation and a marked reduction in ghosting or blurring artifacts compared to traditional deep-learning enhancers.
Conclusion and Future Work
WaveLUT demonstrates that 4D LUTs, when guided by wavelet priors and semantic text-driven loss, can bridge the gap between high-quality enhancement and real-time performance. However, the authors note that in extremely dark scenes with near-zero visual cues, constructing an accurate lighting prior remains challenging. Future research will focus on increasing robustness for these extreme edge cases and further optimizing the framework for deployment on lower-end mobile hardware.
Summary updated