LITA-GS: Illumination-Agnostic Novel View Synthesis via Reference-Free 3D Gaussian Splatting and Physical Priors

Han Zhou, Wei Dong, Jun Chen

Abstract

Directly employing 3D Gaussian Splatting (3DGS) on images with adverse illumination conditions exhibits considerable difficulty in achieving high-quality normally-exposed representation due to: (1) The limited Structure from Motion (SfM) points estimated in adverse illumination scenarios fail to capture sufficient scene details; (2) Without ground-truth references, the intensive information loss, huge noise, and color distortion poses substantial challenges for 3DGS to produce high-quality results; (3) Combining existing exposure correction methods with 3DGS can not achieve satisfactory performance due to their individual enhancement process, which leads to the illumination inconsistency between enhanced images from different viewpoints. To address these issues, we propose LITA-GS, a novel illumination-agnostic novel view synthesis method via reference-free 3DGS and physical priors. Firstly, we introduce an illumination-invariant physical prior extraction pipeline. Secondly, based on the extracted robust spatial structure prior, we develop the lighting-agnostic structure rendering strategy, which facilitates the optimization of the scene structure and object appearance. Moreover, a progressive denoising module is introduced to effectively surpass the noise within the light-invariant representation. We adopt the unsupervised strategy for the training of LITA-GS and extensive experiments demonstrate that LITA-GS surpass the state-of-the-art (SOTA) NeRF-based method by 1.7 dB in PSNR and 0.09 in SSIM while enjoying faster inference speed and costing reduced training time. The code is released at https://github.com/LowLevelAI/LITA-GS.

TL;DR

LITA-GS is a novel framework that enables high-quality 3D Gaussian Splatting (3DGS) in extreme lighting conditions—like near-total darkness or blinding glare—without requiring "perfect" ground-truth images for training. By combining physics-based illumination priors with a progressive denoising module, it reconstructs consistent 3D scenes that are both visually clear and geometrically accurate, outperforming existing NeRF-based methods in both quality and speed.

What Problem Does This Paper Address?

Creating a 3D model of a scene from 2D images (Novel View Synthesis) usually requires well-lit, clear photos. When images are under-exposed (too dark) or over-exposed (too bright), standard methods like 3D Gaussian Splatting (3DGS) and Neural Radiance Fields (NeRF) fall apart for three main reasons:

  1. Structural Failure: Algorithms used to estimate camera positions (Structure from Motion or SfM) struggle to find enough "key points" in the dark or in washed-out highlights, leading to sparse, inaccurate 3D foundations.
  2. Information Loss & Noise: Adverse lighting introduces heavy grain (noise) and color distortion. Without a "clean" reference image to compare against, models often learn to reconstruct the noise rather than the scene.
  3. 3D Inconsistency: If you simply "brighten" images one by one before processing them, the lighting won't match perfectly between different angles. This leads to "flickering" or ghosting artifacts when you try to move through the 3D space.

Methods

The authors introduce LITA-GS (Illumination-Agnostic 3DGS), which uses a "reference-free" approach—meaning it learns directly from the bad images. Key components include:

  • Illumination-Invariant Physical Priors: Drawing on the Kubelka-Munk theory (a mathematical model for how light interacts with surfaces), the authors extract a "structure prior." This is essentially an edge map that remains constant whether the scene is dark or bright. They use this to guide the 3D Gaussians to the correct shapes.
  • Lighting-Agnostic Rendering: Each 3D Gaussian is given extra attributes: an illumination-invariant feature, a depth value, and a noise representation. This allows the model to "untangle" the permanent geometry of the object from the temporary (and messy) lighting conditions.
  • Progressive Denoising Module (PDM): To handle the heavy grain in low-light shots, the authors developed a three-stage CNN module. It uses a "bootstrapping" technique to estimate noise and subtract it from the final render, resulting in clean, smooth surfaces.
  • Unsupervised Optimization: Since there are no "perfect" photos to use as a target, the model uses an Exposure Control Loss. It calculates the average intensity of the input and modulates it toward a target brightness level while maintaining the structural details found in the physical priors.

Datasets

The study primarily uses the LOM (Low-light and Over-exposure Multi-view) dataset.

  • Content: Five real-world scenes: "buu" (a toy), "chair", "sofa", "bike", and "shrub".
  • Hardware: Captured using a DJI Osmo Action 3 camera.
  • Variety: Each scene includes 25 to 48 images captured under challenging lighting (low light and overexposure) alongside "normal" light views used strictly for evaluation (not training).
A normal light reference image of the chair scene used for evaluation.

Key Insights and Ideas

  • Physics over Pure Deep Learning: Instead of asking a neural network to "guess" what a dark room looks like, the authors use the physics of light reflection to identify features that cannot change with exposure. This makes the model much more robust.
  • Decomposition is Key: By treating a pixel as a product of Reflectance × Illumination + Noise, the model can optimize each part separately. This ensures that as you move the camera, the "Reflectance" (the 3D object) stays the same, even if the "Illumination" changes.
  • Efficiency: Because it builds on 3D Gaussian Splatting rather than the ray-tracing used in NeRF, it is significantly faster. It achieves "real-time" rendering speeds while being easier to train.

Key Results

  • Superior Quality: On low-light benchmarks, LITA-GS outperformed the previous state-of-the-art (Aleth-NeRF) by approximately 1.27 dB in PSNR (a measure of image reconstruction quality) and showed a 0.075 improvement in SSIM (structural similarity).
  • Speedup: LITA-GS completes training in roughly 15 minutes on a single NVIDIA RTX 3090. NeRF-based competitors often require hours of training for similar scenes.
  • Visual Fidelity: Qualitative results show that LITA-GS preserves fine textures (like the fabric of a sofa or the leaves of a shrub) that other methods either blur out or lose in the shadows.
Overview of the LITA-GS pipeline and qualitative comparison against state-of-the-art methods.

Limitations and Future Work

  • Monocular Depth Reliance: The method incorporates depth information from a pre-trained model (Marigold). If the depth model fails to interpret a particularly strange shape in the dark, LITA-GS might inherit those geometric errors.
  • SfM Bottleneck: While the rendering is robust, the initial step of estimating camera poses (SfM) still relies on existing tools like COLMAP. In extremely dark environments where no features are visible at all, the system may still struggle to initialize.
  • Future Directions: The authors suggest exploring more complex light-material interactions beyond the Kubelka-Munk model, such as handling highly reflective (specular) surfaces or moving light sources.

Summary updated

Abstract

Directly employing 3D Gaussian Splatting (3DGS) on images with adverse illumination conditions exhibits considerable difficulty in achieving high-quality normally-exposed representation due to:(1) The limited Structure from Motion (SfM) points estimated in adverse illumination scenarios fail to capture sufficient scene details;(2) Without ground-truth references, the intensive information loss, huge noise, and color distortion poses substantial challenges for 3DGS to produce high-quality results;(3) Combining existing exposure correction methods with 3DGS can not achieve satisfactory performance due to their individual enhancement process, which leads to the illumination inconsistency between enhanced images from different viewpoints. To address these issues, we propose LITA-GS, a novel illumination-agnostic novel view synthesis method via reference-free 3DGS and physical priors. Firstly, we introduce an illumination-invariant physical prior extraction pipeline. Secondly, based on the extracted robust spatial structure prior, we develop the lighting-agnostic structure rendering strategy, which facilitates the optimization of the scene structure and object appearance. Moreover, a progressive denoising module is introduced to effectively surpass the noise within the light-invariant representation. We adopt the unsupervised strategy for the training of LITA-GS and extensive experiments demonstrate that LITA-GS surpass the state-of-the-art (SOTA) NeRF-based method by 1.7 dB in PSNR and 0.09 in SSIM while enjoying faster inference speed and costing reduced training time. The code is released at https: //github.com/LowLevelAI/LITA-GS.
BibTeX
@inproceedings{zhou2025lita,
  author = {Han Zhou and Wei Dong and Jun Chen},
  title = {{LITA-GS: Illumination-Agnostic Novel View Synthesis via Reference-Free 3D Gaussian Splatting and Physical Priors}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  url = {https://openreview.net/forum?id=UhcQWLNgue},
}
Figure