Gaussians Can Subtract: Neural Dissolve Fields for Hybrid Scene Rendering

Yiyan Wang, Haohan Liu, Tetsuo ShibuyaarXiv 2026 · arXiv

TL;DR

Gaussolve introduces a "subtract-and-refill" approach to hybrid scene rendering, combining the efficiency of textured meshes with the flexibility of 3D Gaussian Splatting (3DGS). By using a Neural Dissolve Field, the system selectively erases inconsistent mesh content and refills those gaps with residual Gaussians, achieving higher visual fidelity with significantly fewer primitives and faster rendering speeds.

Figure 1. Gaussians Can Subtract. Existing hybrids primarily compensate for mesh errors by adding Gaussian residuals. Gaussolve instead introduces a subtractive Neural Dissolve Field that selectively dissolves inconsistent mesh content and lets residual Gaussians refill the revealed regions. This explicit subtract-and-refill decomposition improves visual quality while retaining a compact hybrid representation. Compared with existing methods, Gaussolve achieves a more favorable quality–compactness trade-off by allowing meshes and Gaussians to cooperate rather than compete.

What Problem Does This Paper Address?

While 3D Gaussian Splatting (3DGS) has revolutionized real-time novel-view synthesis, it is computationally expensive to represent large, flat surfaces (like walls or roads) using thousands of tiny Gaussian primitives. Hybrid methods attempt to solve this by using a textured mesh for the "heavy lifting" of regular geometry and Gaussians for the fine details.

However, existing hybrids are primarily additive: they treat the mesh as a fixed, opaque base and simply layer Gaussians on top. This creates a fundamental conflict. Meshes often contain errors—geometry misalignments, texture-projection artifacts, or incorrect occlusions—that vary depending on your viewing angle. If the mesh is wrong, simply adding Gaussians doesn't fix the underlying "bad" pixels. Furthermore, treating the mesh as a hard, opaque occluder blocks the "learning signal" (gradients) from reaching Gaussians behind the surface, making it nearly impossible for the model to correct errors.

Methods

The authors propose Gaussolve, which reinterprets the mesh-Gaussian conflict as a multi-view inconsistency problem. The framework consists of three main components:

  1. Textured Mesh Branch: Provides a compact structural prior for broad, regular regions.
  2. Neural Dissolve Field (NDF): This is the "subtraction" mechanism. It is parameterized by scalar spherical-harmonic (SH) Gaussians. Unlike standard Gaussians that emit color, these "dissolve Gaussians" predict a view-dependent value that determines how much of the mesh should be "erased" or attenuated from a specific direction.
  3. Residual RGB Gaussians: These "refill" the regions revealed by the NDF, handling fine details, thin structures, and complex view-dependent effects.
Figure 2. Overview of Gaussolve. The projection-textured mesh represents broad regular content, while the Neural Dissolve Field selectively suppresses mesh content that becomes inconsistent across views. Residual Gaussians refill the revealed regions, and all components are combined through continuous transmittance fusion.

To make this work, the authors replace traditional "hard-Z" visibility (where a mesh is either 100% opaque or 0% there) with a Continuous Transmittance Fusion rule. This allows the mesh to become partially transparent, ensuring that reconstruction gradients can flow to the Gaussians behind it during training, enabling them to "learn" how to fill the gaps.

Datasets

The authors evaluated Gaussolve on several standard benchmarks for scene reconstruction:

  • ML-Hypersim-AI: A dataset of highly structured indoor scenes.
  • Deep Blending: A collection of semi-structured indoor and outdoor scenes.
  • Mip-NeRF 360: Used for "stress tests" on unstructured outdoor scenes (like trees and bushes) to find the limits of the mesh-based approach.

Key Insights and Ideas

  • Subtraction is as vital as addition: In hybrid rendering, the mesh isn't just a foundation; it's often a source of error. The ability to "dissolve" the mesh is what allows the Gaussians to actually do their job of correcting the scene.
  • Errors are view-dependent: A mesh might look perfect from one angle but show a stretched texture or a "ghosting" artifact from another. By using scalar SH Gaussians for the NDF, Gaussolve can selectively hide the mesh only when viewed from problematic angles.
  • Soft visibility solves "gradient starvation": Hard visibility boundaries are the enemy of differentiable rendering. By using a soft transmittance model, the system avoids the "Hard-Z" brittleness that prevents Gaussians from crossing the mesh boundary during optimization.
Figure 3. Hard-Z brittleness and gradient recovery. (a) Gaus- sians initially in front of the mesh frequently cross behind the Hard-Z visibility boundary during training. (b) For Gaussians whose reconstruction gradients are blocked by Hard-Z, Gaussolve recovers a substantial fraction of the ungated reference gradient. Curves and shading denote the four-scene mean and range.

Key Results

Gaussolve demonstrates a superior quality-to-compactness trade-off compared to both pure 3DGS and previous hybrid methods:

  • Visual Quality: Achieves higher PSNR and lower LPIPS (perceptual error) than baselines like MeshGS and 3D-GES. On the Deep Blending dataset, it improved average PSNR by ~0.7 dB over existing hybrids.
  • Efficiency: Reduces the Gaussian primitive count by approximately 60% compared to standard 3DGS while maintaining or improving quality.
  • Rendering Speed: Thanks to the reduced Gaussian count, it achieves significant speedups, reaching 174 FPS on an NVIDIA RTX 5000, compared to 99 FPS for standard 3DGS.
  • Versatility: The framework is "backbone-agnostic," meaning it can be plugged into different Gaussian variants like 3DGS-MCMC or Reduced 3DGS to further enhance performance.

Limitations and Future Work

The primary limitation of Gaussolve is its reliance on a decent mesh prior. In highly unstructured scenes (like dense foliage or tangled branches), the initial mesh extraction often fails or produces a "soup" of triangles. In these cases, the mesh and Gaussian branches may compete to explain the same image data, leading to fewer efficiency gains.

The authors suggest that future work could involve geometry-aware routing, which would automatically decide which parts of a scene are best represented by a mesh versus Gaussians, even in highly irregular environments.

Summary updated

Abstract

Hybrid Mesh-Gaussian rendering combines compact textured surfaces with flexible 3D Gaussian primitives, offering an efficient alternative to purely Gaussian scene representations for structured environments. Existing scene-reconstruction hybrids commonly follow an additive formulation, where Gaussian residuals compensate for errors left by the mesh. However, not every Mesh-Gaussian conflict is caused by missing content: residual geometry errors, occlusion mismatch, texture-projection artifacts, and view-dependent appearance can make the same mesh content consistent in one view but inconsistent in another. We therefore formulate hybrid rendering as a multi-view inconsistency problem and present Gaussolve, a subtract-and-refill framework that selectively dissolves inconsistent mesh content before residual Gaussians refill the revealed regions. We introduce a lightweight Neural Dissolve Field, parameterized by scalar spherical-harmonic Gaussians, to learn where and from which viewing directions mesh content should be dissolved. A continuous transmittance fusion rule progressively exposes residual Gaussians behind the dissolved mesh, preserving their reconstruction gradients and avoiding hard visibility discontinuities. Experiments on structured and semi-structured scenes demonstrate higher visual fidelity with fewer Gaussian primitives and faster rendering. Project page: https://ooosn.github.io/gaussians-can-subtract/
BibTeX
@misc{wang2026gaussians,
  author = {Yiyan Wang and Haohan Liu and Tetsuo Shibuya},
  title = {{Gaussians Can Subtract: Neural Dissolve Fields for Hybrid Scene Rendering}},
  year = {2026},
  eprint = {2611.07049},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.07049},
}
Figure