TL;DR
ClearForm is a novel 3D Gaussian Splatting (3DGS) framework designed to accurately reconstruct both transparent and opaque surfaces in a single scene. By decoupling the way geometry and appearance are calculated—solving what the authors call the "Geometry-Appearance Compositing Dilemma"—it achieves state-of-the-art mesh accuracy and rendering quality for complex glass objects.
What Problem Does This Paper Address?
Reconstructing transparent interfaces, like glass windows or laboratory beakers, is a notorious challenge in computer vision. Most of the light we see at a "glass" pixel actually comes from the scene behind the glass.
Standard 3DGS methods use a single "opacity" (alpha) value for each Gaussian primitive. This alpha controls two things simultaneously: how much color the primitive contributes to the image and how much it contributes to the geometric depth calculation. This creates the Geometry-Appearance Compositing Dilemma (GACD): to make a surface look transparent, you must lower its alpha, but doing so also weakens its geometric signal. Consequently, the reconstructed "surface" often gets pulled toward the background or disappears entirely.
Methods
ClearForm introduces three key innovations to break this dilemma:
- Dual-Opacity Compositing: Instead of one alpha, ClearForm uses two separate rendering paths. The geometry path uses the Gaussian's full opacity to define the surface boundary. The appearance path attenuates that same alpha based on a learned transparency value ($\tau$), allowing background light to pass through without weakening the geometric reconstruction.
- Dual-Evidence Lifecycle (DEL): Standard 3DGS prunes "useless" primitives based on low opacity. However, a Gaussian might have low optical opacity (it's very clear glass) but be vital for geometry. DEL retains primitives that provide either geometric visibility or a positive contribution to the RGB reconstruction across multiple views.
- Transparency-Aware Regularization: The authors adapt geometric constraints (like depth-normal consistency) to specifically handle transparent regions. This prevents the model from mistakenly trying to "project" the texture of background objects onto the foreground glass surface.
Datasets
The authors evaluated ClearForm on three primary datasets:
- 3D-FRONT-T: A synthetic dataset of indoor scenes featuring transparent furniture and glass partitions.
- TransLab: A challenging benchmark of laboratory glassware (beakers, flasks) with thin, curved surfaces.
- Ref-DL3DV: A subset of the DL3DV-10K dataset containing real-world scenes with significant reflections and transparency.
Key Insights and Ideas
- The "Aha!" Moment: The realization that the alpha value in 3DGS is "overloaded." By mathematically deriving how shared alpha biases depth toward the background, the authors proved that decoupling is a necessity, not just a luxury, for transparent scenes.
- Pruning is a Mismatch: Traditional pruning strategies are biased toward opaque objects. The Dual-Evidence Lifecycle recognizes that "importance" in a scene is multi-faceted—some primitives are there to define shape, while others are there to refine the view-dependent look of what's behind the glass.
- Shared Primitives, Different Roles: Unlike some methods that use entirely different sets of Gaussians for glass vs. background, ClearForm uses shared planar Gaussians that simply behave differently in the two compositing paths. This maintains a compact and efficient representation.
Key Results
ClearForm significantly outperforms previous state-of-the-art methods in both geometry and rendering:
- 3D-FRONT-T: It achieved a 27.05 mm Chamfer Distance (CD), a massive improvement over the 59.18 mm of the base PGSR model and the 34 mm reported by GLINT. Its F1 score (0.928) also set a new benchmark.
- TransLab: ClearForm reached a CD of 1.65 mm and a PSNR of 40.14 dB, producing highly coherent vessel surfaces where other methods produced noisy or incomplete meshes.
- Qualitative Superiority: As shown in the figures, ClearForm recovers sharp, accurate normals for glass objects while perfectly preserving the appearance of the scene visible through them.
Limitations and Future Work
While a major step forward, the authors note several areas for improvement:
- Dependency on Priors: The method relies on transparency masks (priors), which can sometimes be inaccurate or missing in complex real-world scenes.
- Reflection Bias: Extremely strong reflections in regions labeled as "opaque" can still occasionally bias the depth estimation.
- Mesh Extraction: The current TSDF fusion process for extracting the final mesh can sometimes "cap" open containers (like the top of a beaker) because it only sees a single depth layer per ray. Future work could involve multi-layer mesh extraction to handle complex topologies better.
Summary updated