EAG-PT: Emission-Aware Gaussians and Path Tracing for Indoor Scene Reconstruction and Editing

Xijie Yang, Mulin Yu, Changjian Jiang, Kerui Ren, Tao Lu, Jiangmiao Pang, Dahua Lin, Bo Dai, Linning Xu

Abstract

Recent reconstruction methods based on radiance fields, such as NeRF and 3DGS, reproduce indoor scenes with high visual fidelity but fail during scene editing due to baked illumination and a lack of explicit light transport. Physically based inverse rendering typically relies on mesh representations and path tracing, which enforce correct light transport but are limited by the geometric fidelity of reconstructed meshes. We propose Emission-Aware Gaussians and Path Tracing (EAG-PT), a framework for physically based light transport using a unified 2D Gaussian representation. By explicitly separating emissive and non-emissive components and decoupling reconstruction from multi-bounce path tracing, EAG-PT produces physically consistent renders after editing while preserving fine geometric details.

Introduction: The Challenge of Baked Illumination

Modern 3D reconstruction techniques like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) excel at novel view synthesis but struggle with scene editing. Because these methods "bake" the lighting directly into the scene's appearance, modifying the environment—such as changing a light's color or moving an object—results in unrealistic visuals that lack proper shadows and interreflections.

Traditional solutions involve inverse rendering to extract a 3D mesh for use in path tracing. However, extracting high-quality meshes from cluttered real-world scenes is a significant bottleneck. Errors in the mesh geometry, such as jagged edges on thin structures, lead to severe artifacts in light simulation. EAG-PT addresses this by asking: Can we perform physically accurate path tracing directly on a Gaussian representation, bypassing the need for meshes entirely?

The original captured scene used for reconstruction and editing experiments, featuring complex geometry and lighting.

Methodology: Mesh-Free Path Tracing with 2D Gaussians

EAG-PT introduces a pipeline that bypasses the mesh bottleneck by performing path tracing directly on 2D Gaussians. Unlike their 3D counterparts, 2D Gaussians act as flat surface elements with well-defined normals, making them ideal proxies for ray-surface intersections.

The framework operates in three distinct phases:

  1. Radiant Scene Reconstruction: The scene is reconstructed using 2D Gaussians. Crucially, emissive light sources are explicitly masked and separated from non-emissive surfaces (e.g., walls and furniture).
  2. Material Recovery: To determine the true diffuse albedo of surfaces, the system uses a single-bounce differentiable renderer. It leverages the initial radiant field as a "radiance cache" to approximate incoming light efficiently during training, avoiding the cost of multi-bounce simulations.
  3. Path Tracing and Light Baking: After a user edits the scene, EAG-PT performs multi-bounce path tracing on the 2D Gaussians to simulate global illumination. To enable real-time interaction, the path-traced results are "baked" back into the Gaussians' radiance parameters, dropping render times from minutes to milliseconds.

Results and Discussion

EAG-PT significantly outperforms standard radiant scene reconstructions in editing tasks. While basic 3DGS-based editing fails to account for light transport, EAG-PT captures complex physical effects like soft shadows and color bleeding (interreflection).

Comparison of editing results. Standard radiant scene editing (bottom left) lacks shadows and interreflections. EAG-PT (center) provides physically consistent global illumination, including shadows under chairs and light bouncing off walls.

In comparisons with state-of-the-art mesh-based methods like FIPT, EAG-PT demonstrates superior preservation of fine geometric details. Thin structures like lamp arms and chair legs, which often become distorted in mesh reconstructions, remain sharp and visually accurate. Furthermore, the light baking strategy reduces rendering times from minutes per frame to just 0.015 seconds, enabling real-time playback at over 60 FPS with virtually no loss in visual quality.

Conclusion

EAG-PT provides a robust framework for physically accurate indoor scene editing without the need for error-prone mesh reconstruction. By combining the flexibility of 2D Gaussians with the physical rigor of path tracing, it opens new possibilities for interior design, XR content creation, and embodied AI. Future work will focus on extending the model to handle complex non-diffuse materials and automating the emission masking process.

Summary updated

Abstract

Recent reconstruction methods based on radiance field such as NeRF and 3DGS reproduce indoor scenes with high visual fidelity, but break down under scene editing due to baked illumination and the lack of explicit light transport. In contrast, physically based inverse rendering relies on mesh representations and path tracing, which enforce correct light transport but place strong requirements on geometric fidelity, becoming a practical bottleneck for real indoor scenes. In this work, we propose Emission-Aware Gaussians and Path Tracing (EAG-PT), aiming for physically based light transport with a unified 2D Gaussian representation. Our design is based on three cores: (1) using 2D Gaussians as a unified scene representation and transport-friendly geometry proxy that avoids reconstructed mesh, (2) explicitly separating emissive and non-emissive components during reconstruction for further scene editing, and (3) decoupling reconstruction from final rendering by using efficient single-bounce optimization and high-quality multi-bounce path tracing after scene editing. Experiments on synthetic and real indoor scenes show that EAG-PT produces more natural and physically consistent renders after editing than radiant scene reconstructions, while preserving finer geometric detail and avoiding mesh-induced artifacts compared to mesh-based inverse path tracing. These results suggest promising directions for future use in interior design, XR content creation, and embodied AI.
BibTeX
@inproceedings{yang2026eag,
  author = {Xijie Yang and Mulin Yu and Changjian Jiang and Kerui Ren and Tao Lu and Jiangmiao Pang and Dahua Lin and Bo Dai and Linning Xu},
  title = {{EAG-PT: Emission-Aware Gaussians and Path Tracing for Indoor Scene Reconstruction and Editing}},
  booktitle = {SIGGRAPH (Conference Paper Track)},
  year = {2026},
  doi = {10.1145/3799902.3811054},
}
Figure