Capturing high-quality images in near-darkness is a double-edged sword. Not only is the signal weak and buried in noise, but the artificial light sources common in low-light environments (like streetlights or indoor LEDs) often introduce strong, varied color casts. Traditionally, researchers have treated brightening the image and correcting its color as two separate problems. A new paper, CHROMA-MILL, argues that this "divide and conquer" approach is fundamentally flawed and proposes a joint solution.
TL;DR
The authors introduce CHROMA-MILL, a novel RAW dataset for joint low-light enhancement and white balance, and CTI-Retinexformer, a deep learning model that disentangles intensity, color, and scene content. Their findings show that joint correction significantly outperforms traditional cascaded pipelines by preventing error propagation between stages.
What Problem Does This Paper Address?
In the standard imaging pipeline, Low-Light Image Enhancement (LLIE) (brightening and denoising) and Auto White Balance (AWB) (removing color casts) are usually performed sequentially. Typically, LLIE happens first. However, AWB algorithms rely on cues like "gray" surfaces or bright highlights to estimate the scene's illuminant. In low light, these cues are corrupted by noise.
If you enhance first, you might "bake in" color errors. If you white balance first, the noise makes the estimate unreliable. This "chicken-and-egg" problem has been hard to study because existing low-light datasets (like LOL or SID) were captured under neutral lighting and lack the ground truth needed to evaluate AWB performance.
Methods
The authors propose CTI-Retinexformer, a joint architecture designed to solve both tasks at once. It builds on the "Retinexformer" backbone but introduces a sophisticated latent space disentanglement strategy.
The model's encoder maps the input into three distinct components:
- Intensity ($Z_I$): A single channel representing the illumination map.
- Chromaticity ($Z_{ch}$): Two channels predicting the CIE xy chromaticity of the scene illuminant.
- Scene Content ($Z_s$): The remaining channels, which are trained to be invariant to both brightness and color cast.
To enforce this disentanglement, they use a suite of losses, including a triplet margin loss that encourages the scene embedding ($Z_s$) to stay the same even if the lighting color or intensity changes. This ensures the model doesn't accidentally store color information in the "content" part of its brain.
Datasets
The core of this work is the CHROMA-MILL dataset. It consists of 50 scenes captured with a DSLR and a smartphone under four calibrated illuminants (Red, Green, Blue, and White) at 11 different intensity levels.
A key innovation here is the use of linear-RAW blending. Because light is additive in the linear RAW domain, the authors can mathematically mix their captured images to synthesize thousands of "physically valid" illuminant mixtures. This provides a dense coverage of the chromaticity plane with mathematically exact ground truth for the illuminant color.
Key Insights and Ideas
- Cascades Compound Errors: The systematic benchmark reveals that neither "LLIE-first" nor "AWB-first" is ideal. Whichever module runs first often hands the second module an "out-of-distribution" input, leading to visible artifacts and residual color casts.
- Joint is Better: A single network sharing features across both tasks avoids the problematic interface of a cascade.
- Disentanglement is the Secret Sauce: By explicitly supervising the chromaticity and intensity in the latent space, the model learns a more robust representation of the underlying scene. Figure 6 in the paper shows that the chromaticity code ($Z_{ch}$) responds sharply to light changes while remaining indifferent to scene changes, and vice versa for the content code ($Z_s$).
Key Results
Quantitatively, CTI-Retinexformer sets a new state-of-the-art on the CHROMA-MILL benchmark.
- Performance: It outperforms the best-performing cascaded pipeline by 0.7 dB PSNR$_L$ (luminance fidelity) and improves color accuracy by 1.1 $\Delta E_{00}$.
- Qualitative Superiority: As seen in Figure 4, while cascaded methods often leave a blue or green tint or over-correct whites into "past-neutral" territory, the joint model produces much cleaner, more natural results.
- Generalization: Despite being trained only on CHROMA-MILL, the model generalizes well to other datasets like DICM and LOLv1 without any fine-tuning.
Limitations and Future Work
While a significant step forward, the authors note a few limitations:
- Uniform Lighting: The current model assumes a spatially uniform illuminant. Real-world scenes often have mixed lighting (e.g., a warm lamp and cool moonlight), which would require per-pixel chromaticity estimation.
- Gamut Constraints: The LEDs used for the dataset don't cover extremely warm light sources (like candlelight).
- Dataset Size: 50 scenes is relatively small for deep learning, though the synthetic blending helps mitigate this.
Future work could involve extending the blending technique to synthesize spatially varying lighting and expanding the dataset to include a wider variety of light sources.
Summary updated