3D Gaussian Splatting Model for Low-Light Enhancement

L.m. Cheng, Changhao Li, Ruizhen Hu, Ligang LiuJournal of Image and Graphics 2025 · DOI

TL;DR

This paper introduces Enhanced-GS, a novel framework that brings high-quality, real-time low-light enhancement to 3D Gaussian Splatting (3DGS). By decoupling intrinsic scene color from lighting and employing a "fixed geometry" optimization strategy, the authors achieve state-of-the-art image quality while rendering 1,000x faster than previous NeRF-based low-light methods.

What Problem Does This Paper Address?

Novel View Synthesis (NVS) has been revolutionized by Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS). However, these models typically assume well-lit environments. When trained on low-light images, they produce dark, noisy renders that are useless for downstream tasks like robotics or AR.

Existing solutions, such as Aleth-NeRF, attempt to "relight" the scene during training. While effective, they suffer from two major drawbacks:

  1. Speed: Being NeRF-based, they are computationally expensive, taking hours to train and seconds to render a single frame.
  2. Detail Loss: Low-light enhancement often amplifies noise, which confuses the model's geometry optimization, leading to blurry high-frequency details (like text or fine textures).

The authors set out to create a model that is both fast (real-time) and robust enough to recover sharp details from the shadows.

Methods

The core of the proposed Enhanced-GS method lies in two clever design choices:

1. Color-Lighting Decomposition

Inspired by Retinex theory (which posits that an image is the product of illumination and reflectance), the authors decompose the color attribute of each 3D Gaussian.

  • Instead of storing a single color, the model learns an intrinsic color ($r$) and a lighting factor ($v$).
  • A lightweight, 4-layer MLP (Multi-Layer Perceptron) predicts the local lighting $v$ based on the Gaussian's spatial properties and view direction.
  • The Magic Trick: During training, the model renders the "low-light" version ($v \times r$) to match the input. At inference, it simply renders the intrinsic color $r$, effectively "turning on the lights" in the scene.

2. Fixed Geometry Optimization

In standard 3DGS, the model constantly adds, moves, or prunes Gaussians based on image gradients. In low light, enhanced image gradients are extremely noisy. To prevent this noise from ruining the scene's structure, the authors use a dual-gradient approach:

  • Geometry (Position, Opacity, Scale): Updated using gradients from the original low-light images, which provide more stable structural cues.
  • Appearance (Color, Spherical Harmonics): Updated using gradients from the enhanced images to ensure accurate color recovery.

3. Specialized Loss Functions

The model uses a suite of losses to ensure visual fidelity:

  • Inverse Tone Mapping MSE: Corrects for the non-linear nature of low-light sensors.
  • Exposure Loss: Guides the model toward a target global brightness.
  • Color Constancy Loss: Based on the "Gray World" assumption to prevent unnatural color shifts (e.g., the scene looking too blue or yellow).

Datasets

The authors evaluated their model on two primary benchmarks:

  • LOM Dataset: A multi-view dataset featuring scenes captured under low-light, normal-light (ground truth), and over-exposed conditions. It includes 5 distinct scenes (Bike, Buu, Chair, Shrub, Sofa).
  • LLNeRF Dataset: A collection of real-world low-light scenes without ground truth, used to test the model's performance in "wild" scenarios.

Key Insights and Ideas

  • Structure is more stable in the dark: One of the most interesting findings is that raw low-light images are better for optimizing geometry than enhanced ones. Artificially brightening an image introduces "hallucinated" noise that causes 3DGS to create "floaters" or artifacts.
  • Decoupling enables zero-cost relighting: By baking the lighting into a separate MLP-predicted factor, the model doesn't need to perform complex calculations at render time to show the "bright" version; it just skips the multiplication step.
  • Efficiency doesn't require sacrifice: Moving from NeRF's volume rendering to 3DGS's point-based splatting provides a massive speedup without losing the ability to perform complex color decomposition.

Key Results

Enhanced-GS outperforms current state-of-the-art (SOTA) methods like Aleth-NeRF across the board:

  • Image Quality (LOM Dataset):
    • PSNR: Improved by 0.12 dB.
  • SSIM: Improved by 1.3%.
  • LPIPS (Perceptual Quality): Improved by 5.5%.
  • Efficiency:
    • Training Time: Reduced from ~6 hours to just 30 minutes (a 12x speedup).
  • Rendering Speed: Achieved 268 FPS, compared to Aleth-NeRF's 0.22 FPS—a staggering 1,200x increase.
  • Qualitative: Visual results show significantly sharper text and thinner structures (like bicycle spokes) compared to NeRF-based methods, which tended to blur these areas.

Limitations and Future Work

While a major step forward, the authors note a few remaining hurdles:

  • Sparse Inputs: Like standard 3DGS, the model struggles when very few input images are provided, leading to "holes" or artifacts at the edges of the scene.
  • Extreme Noise: In near-zero light conditions, some sensor noise can still be "baked" into the Gaussian colors.
  • Future Directions: The authors suggest exploring generative priors (like Diffusion Models) to help "hallucinate" missing high-frequency details in extremely degraded images and implementing adaptive initialization to handle sparse viewpoints better.

Summary updated

Abstract

目的使用低光图像训练神经渲染模型时,无法合成正常光照条件下的新视角图像。且低光照条件下处理目标检测、语义分割等任务时,模型性能会明显下降,面临较大的挑战。此外,现有方法在渲染速度和图像高频细节方面存在不定。针对现有问题,本文提出一种对三维高斯泼溅模型进行低光增强的方法。方法首先利用一个轻量化的光照预测网络将三维高斯泼溅模型中三维高斯分布的颜色属性分解为物体本征颜色和光照两个部分,利用本征颜色渲染得到正常光照场景图像,同时使用多种损失函数从结构和颜色上改善图像质量;为了提高图像中高频细节的清晰度,采用固定几何的优化方案。结果实验在低光场景的新视角合成数据集LOM(low-light & normal-light & over-exposure multi-view dataset)上与主流方法进行了比较,与对比结果中最佳方法相比,峰值信噪比(peak signal-to-noise ratio,PSNR)指标平均提升0.12 dB,结构相似性(structural similarity index, SSIM)指标平均提升1.3%,学习感知图像块相似度(learned perceptual image patch similarity, LPIPS)指标平均提高5.5%,训练时间仅有以往方法的1/5,渲染速度则达到以往方法的1 000倍以上。结论本文方法能够更快地进行训练和渲染,同时也具有更高的图像质量,图像的高频细节和结构更加清晰,并通过全面的对比实验验证了方法的有效性与先进性。
BibTeX
@article{cheng2025d,
  author = {L.M. Cheng and Changhao Li and Ruizhen Hu and Ligang Liu},
  title = {{3D Gaussian Splatting Model for Low-Light Enhancement}},
  journal = {Journal of Image and Graphics},
  year = {2025},
  doi = {10.11834/jig.240598},
}
Figure