Fast 3D reconstruction systems now build a scene in a single forward pass of a neural network. Instead of optimizing each scene for minutes, they predict small 3D blobs called Gaussians, one or more per pixel of each input photo. When several photos overlap, the same physical surface is predicted many times over. Recent methods remove these duplicates by merging blobs that look alike and discarding blobs in low-density space. Both rules quietly assume opaque surfaces. A glass beaker shows a different refracted background in every view, so its blobs do not look alike across views. To render as see-through, it also has to be represented by nearly invisible, low-opacity blobs. The usual deduplication rules therefore either leave the glass as a stack of conflicting copies or delete it as empty space. A further problem is that the cues that reveal glass, such as thin rims, contact edges and one-pixel refraction boundaries, are exactly what image downsampling blurs away before any merging happens.
We propose to decide which blobs are "the same surface" and which blobs "deserve to survive" from view-consistent geometric evidence rather than from appearance or opacity. The model learns a geometry-only identity feature with an explicit depth-layer index, so that a glass shell and the liquid behind it are never merged. It also learns a survival score driven by full-resolution boundary and surface-normal cues, so that a faint but well-supported glass surface is kept. The claim we test is narrow. At a matched primitive budget, this fusion rule recovers transparent surfaces that appearance-keyed and density-keyed fusion lose, without costing quality on ordinary opaque scenes.
Core Claim and What Is New
Feed-forward deduplication currently merges with one of three signals. Learning Compact 3D Gaussians via Feed-Forward Point Fusion (SPLATT3RFUSION) merges points inside octree cells by the cosine similarity of learned features. Learning Efficient Fuse-and-Refine for Feed-Forward 3D Gaussian Splatting deposits Gaussians into a voxel grid and keeps only the 20% of voxels with the most feature mass. Gaussian Graph Network pools Gaussians across views linked by field-of-view overlap. FreeSplat fuses through cost volumes that its authors report break on specular and textureless surfaces. These papers supply the machinery we reuse. They also motivate the problem, because every one of them is evaluated only on opaque benchmarks.
On the transparent side, TSGS names the "transparency-depth dilemma": glass Gaussians must have low opacity, so background Gaussians outvote them in rendered depth. TSPE-GS recovers several surfaces along one ray by finding multiple drops in transmittance. Clear-Splatting learns residual Gaussians for glass on top of a background captured separately. All three optimize each scene individually and do no cross-view deduplication. Their findings constrain our design: opacity cannot be the keep signal, and depth along a ray must be allowed to have more than one mode.
What is new is the combination: a feed-forward merge rule that is invariant to refraction-induced appearance change, aware of depth layers, and protective of low-opacity surfaces. Its two ingredients can be ablated separately.
Research Program
The base system is a VGGT-backbone point-fusion pipeline in the style of SPLATT3RFUSION. VGGT is a pose-free transformer that predicts per-pixel 3D points. We change three things.
Two-mode depth per pixel. Each pixel predicts up to two depth hypotheses: a first surface and a surface behind it. This brings TSPE-GS's multi-peak idea into the forward pass instead of applying it as post-processing. Each resulting point carries a layer index.
A geometry identity feature. We split each point's feature into an appearance part and a geometry part. The geometry part is trained contrastively on synthetic multi-view data with known ground-truth surfaces. Two points from different views count as positives when their ground-truth surface location and layer coincide within a small radius, even though their colors differ because of refraction. Points on different layers of the same ray are explicit negatives. Octree merging then uses only the geometry feature, and it never merges across layers.
A survival score. Merged nodes are kept or culled by a learned score. The score is supervised to predict whether a node lies on a ground-truth surface, and it takes as input full-resolution boundary and normal evidence aggregated across views. The boundary evidence follows the argument in ClearGrasp that edges and normals are more reliable than raw depth for glass. Neither opacity nor feature mass is an input to the score.
To keep full-resolution evidence without the memory blow-up that the Gaussian Graph Network authors report at high input resolution, a light side branch runs at native resolution. It uses importance-weighted pooling in the manner of LIP, which raised small-object detection precision on COCO, and it only produces boundary, normal and transparency-mask maps. The main backbone stays at its usual resolution.
Training data are synthetic multi-view renders of glassware, liquids, and objects inside containers placed in textured indoor scenes, plus opaque scenes from RealEstate10K, so that opaque performance is preserved.
Baselines
The do-nothing reference is SPLATT3RFUSION and Fuse-and-Refine run as released on transparent scenes. This measures how much glass current fusion actually loses. There is no reported result found on Lacuna for either method on ClearPose, TransLab, αSurf or Bottleship, so we produce these numbers ourselves.
The strongest feed-forward competitor is SPLATT3RFUSION fine-tuned on exactly our synthetic transparent training set, with the same VGGT backbone, the same GPU-hour budget, and the same hyperparameter search. Its merge threshold is swept so that the two methods are compared at equal Gaussian counts. Without this control, any gain could simply come from the new data. For opaque context, SPLATT3RFUSION reports 40.0k Gaussians against NoPoSplat's 131.1k on RealEstate10K at τ=0.995.
The strongest pipeline competitor is GHOST. GHOST paints an opaque, geometry-consistent texture over glass in each image before reconstruction. Its authors report that DUSt3R reconstruction F-score rises from 77.36 to 88.66 with this preprocessing, and that DepthAnythingV3 depth RMSE falls from 0.023 to 0.018. We run GHOST in front of the fine-tuned SPLATT3RFUSION.
The quality ceiling is per-scene optimization: TSGS on TransLab, and TSPE-GS on αSurf (reported Chamfer distance 0.82) and Bottleship (reported 0.88). TSGS reports a 37.3% Chamfer reduction over PGSR on TransLab, but our file contains no absolute figure for it.
Ablations isolate the key ingredients by swapping one piece at a time. The geometry identity feature is replaced by the full appearance-plus-geometry feature. The survival score is replaced by opacity-mass culling. Two-mode depth is replaced by single depth. The full-resolution branch is removed.
Beating the fine-tuned baseline means higher transparent-surface recall and lower Chamfer distance on glass regions, at equal Gaussian count, across ClearPose and TransLab. It also means opaque RealEstate10K quality stays within 0.3 dB PSNR of the baseline. Beating GHOST additionally requires winning on the nested-transparency scenes, where opacification erases the interior surface by construction.
Why not just…?
Why not just fine-tune an existing compressor on transparent data? This is the most serious threat. If data alone teaches cosine-similarity merging to group refracted views of glass, our mechanism is unnecessary. The fine-tuned SPLATT3RFUSION baseline settles this directly. We also run a pre-registered diagnostic: on held-out synthetic scenes, we compare cross-view feature similarity for corresponding glass points and for corresponding opaque points, for both feature types. If fine-tuned appearance features already separate glass correspondences as well as geometry features do, we will report that the thesis fails.
Why not just keep more primitives, by lowering the sparsification ratio or the merge threshold? If glass disappears only because culling is aggressive, keeping everything would recover it. That is why every comparison is a quality-versus-budget curve, not a single point. Keeping all primitives also does nothing about duplicated, clashing copies of glass, which is the ghosting described by DrivingRecon. We measure that duplication separately as the number of primitives per unit of ground-truth glass area.
Why not just opacify with GHOST and use any opaque pipeline? For thin single-layer glass this may work well, and we expect GHOST to be competitive there. Its weak-refraction assumption, however, removes interior geometry. It also hallucinates texture independently in each frame, with no guarantee of cross-view consistency. We test both weaknesses on Bottleship and on ClearPose's liquid and translucent-cover scenes, where ClearPose reports pose accuracy falling to 13% ADD-S.
Why not just run per-scene TSGS? TSGS takes about 48 minutes per scene, and Clear-Splatting needs a separate background capture. Neither is usable for a robot that sees a scene once. These methods serve as the accuracy ceiling, and we report how much of the gap a single forward pass closes.
Proposed Experiments
The first check is the diagnostic, run before any new training. We take the released SPLATT3RFUSION and Fuse-and-Refine models and apply them to TransLab and ClearPose, using ClearPose's annotated objects for ground-truth geometry. We measure the fraction of ground-truth glass surface lying within a distance threshold of any surviving Gaussian, and we report how many glass voxels fall outside Fuse-and-Refine's top-20% mass cut. If recall on glass matches recall on opaque surfaces, the motivating gap is not real and the project stops there.
The main experiment then compares the method against all baselines on ClearPose, TransLab, αSurf and Bottleship. Metrics are glass-region Chamfer distance and recall, depth RMSE inside the transparent mask, PSNR, and Gaussian count, all plotted as functions of the primitive budget. Opaque regression is checked on RealEstate10K and ScanNet++.
The decisive result has two parts. The full method must hold glass recall flat as the budget shrinks, while the appearance-keyed ablation loses glass first. And the two-layer variant must reduce Bottleship Chamfer distance relative to single-layer fusion. If only the survival score helps and the identity feature does not, the contribution reduces to "do not cull by opacity", and we would report it that way.
References
- Learning Compact 3D Gaussians via Feed-Forward Point Fusion
- Learning Efficient Fuse-and-Refine for Feed-Forward 3D Gaussian Splatting
- Gaussian Graph Network: Learning Efficient and Generalizable Gaussian Representations from Multi-view Images
- FreeSplat: Generalizable 3D Gaussian Splatting Towards Free View Synthesis of Indoor Scenes
- TSGS: Improving Gaussian Splatting for Transparent Surface Reconstruction via Normal and De-lighting Priors
- TSPE-GS: Probabilistic Depth Extraction for Semi-Transparent Surface Reconstruction via 3D Gaussian Splatting
- Clear-Splatting: Learning Residual Gaussian Splats for Transparent Object Manipulation
- Clear Grasp: 3D Shape Estimation of Transparent Objects for Manipulation
- LIP: Local Importance-Based Pooling
- GHOST: Geometry-Guided Hallucination of Opaque Surface Textures
- DrivingRecon: Large 4D Gaussian Reconstruction Model For Autonomous Driving