Low-light video enhancement is a notoriously difficult task in computer vision. Traditional supervised methods require massive, perfectly aligned datasets of "dark" and "bright" video pairs—a resource-intensive requirement that often leads to "domain gaps," where models trained on synthetic data fail to generalize to real-world conditions.
In their AAAI-24 paper, Haofeng Huang and colleagues from Peking University and Peng Cheng Laboratory propose SLBNR (Self-Learned Bottleneck Neural Representation). This approach requires zero external training data, learning to enhance a video by optimizing directly on the noisy, dark input sequence itself.
The Challenge of Low-Light Video
Capturing video in low light typically results in two major issues: extreme noise and "color flicker" caused by unstable white balance. While deep learning has advanced image restoration, applying it to dark videos remains tricky. Supervised methods often fail in authentic dark scenes due to training data limitations. Conversely, existing self-supervised methods often struggle with noise overfitting (e.g., Blind-Spot Networks) or introduce artifacts if frames are not perfectly aligned (e.g., Noise2Noise).
The Joint Bottleneck Framework
The core of the SLBNR solution is a Joint Bottleneck approach that constrains the learning process from two directions: the input content and the optimization objective.
- Hybrid Neural Representation (Content Bottleneck): Instead of relying solely on pixel coordinates, the model uses a compact deep embedding. By restricting this embedding to a low-dimensional space, the network is forced to capture the consistent, intrinsic structure of the scene while ignoring random, high-frequency noise.
- Entropy Minimization (Objective Bottleneck): Noise is inherently chaotic and high-entropy. The authors model the signal distribution using a Gaussian-Mixture Model (GMM) and optimize the network to minimize entropy. This effectively "squeezes out" noise from the final prediction.
As shown in the figure above, neither a content bottleneck nor an objective bottleneck is sufficient on its own. Combining them allows SLBNR to integrate information across multiple frames naturally, leading to superior temporal consistency without heavy-duty alignment.
Chromatic Retinex Decomposition
To address the color flicker common in dark videos, the authors developed a novel decomposition method. Unlike standard Retinex models that assume monochromatic light, this version accounts for chromatic (colored) illumination. By isolating color fluctuations into a specific illumination layer, the model can maintain a temporally consistent "reflectance" layer (the actual objects). This decoupling makes self-supervised warping losses significantly more stable in low-light conditions.
Experimental Results
The authors evaluated SLBNR on the DRV (Dynamic Raw Video) dataset, using real-world dynamic videos of dark scenes. Since ground truth is unavailable for these sequences, the performance was measured using three standard no-reference metrics: NIQE, ILNIQE, and NIQMC.
SLBNR achieved the best scores across all metrics, outperforming unsupervised image methods (Zero-DCE), supervised video methods (SDSD), and unsupervised denoising methods (UDVD).
| Method | NIQE ↓ | ILNIQE ↓ | NIQMC ↑ |
|---|---|---|---|
| Zero-DCE | 5.1637 | 44.0923 | 3.9014 |
| SDSD (Supervised) | 13.0713 | 59.5753 | 3.9162 |
| UDVD (Self-supervised) | 13.5302 | 51.8735 | 2.6228 |
| Ours (SLBNR) | 4.7616 | 31.1489 | 4.8123 |
Qualitatively, SLBNR produces clear frames with natural colors and sharp details, whereas other methods often leave behind heavy noise or create over-smoothed, blurry images.
Conclusion
SLBNR demonstrates that high-quality video enhancement is possible without external training data. By leveraging a joint bottleneck strategy and chromatic Retinex decomposition, the framework effectively suppresses noise and corrects color flicker. While the current optimization process is slower than one-shot supervised models, the ability to generalize to real-world dark scenes makes it a powerful tool for authentic low-light restoration.
Summary updated