TL;DR
PlanarSplatting is an ultra-fast optimization framework that reconstructs accurate 3D planar surfaces of indoor scenes in just three minutes. By directly optimizing rectangular 3D primitives through a novel differentiable splatting function, it eliminates the need for complex plane detection or matching pipelines and significantly boosts the performance of downstream tasks like Gaussian Splatting.
What Problem Does This Paper Address?
Indoor environments are inherently structured, dominated by flat surfaces like walls, floors, and ceilings. While traditional 3D reconstruction methods often use points, voxels, or meshes, representing these scenes with 3D planes is far more compact and structurally expressive.
However, existing planar reconstruction methods face significant hurdles:
- Complexity: They often rely on multi-stage pipelines that must detect, match, and track planes across different views.
- Data Dependency: Many state-of-the-art models require expensive 2D or 3D plane annotations for supervision.
- Speed vs. Detail: Current methods are either too slow for practical use or produce coarse results that miss fine geometric details.
PlanarSplatting aims to bridge this gap by providing a method that is both high-speed and high-fidelity without requiring explicit plane labels.
Methods
The core innovation of PlanarSplatting lies in treating 3D planes as explicit, learnable primitives that can be "splatted" onto an image plane, similar to the popular Gaussian Splatting technique.
1. Learnable Planar Primitives
Instead of points, the authors represent the scene using a collection of 3D rectangles. As shown in Figure 2, each primitive is defined by its center, rotation, and a unique "Double Radii" design (four independent radii for the positive/negative X and Y axes), allowing for flexible rectangular shapes.
2. Differentiable Planar Splatting
The system renders these primitives into 2.5D depth and normal maps. A key contribution is the rectangle-based splatting function. Standard Gaussian splats have "fuzzy" boundaries that don't fit sharp corners well. The authors designed a function that starts fuzzy to allow for easy optimization but gradually sharpens into a crisp rectangle as iterations progress (see Figure 4).
3. Optimization and Supervision
The model is optimized using an Adam optimizer over 5,000 iterations. Since it doesn't use ground-truth plane labels, it leverages "pseudo-labels" from modern monocular foundation models (like Metric3Dv2 for depth and Omnidata for normals). To handle complex geometry, the system includes a splitting and merging strategy: planes with high gradients are split to capture more detail, and similar adjacent planes are merged after optimization to create a clean final model.
Datasets
The authors evaluated their method on two major indoor reconstruction benchmarks:
- ScanNetV2: A large-scale dataset of indoor spaces. The authors used a test split of 100 scenes with ground-truth 3D plane annotations.
- ScanNet++: A higher-fidelity dataset where 30 scenes were randomly selected for evaluation.
Key Insights and Ideas
- Planes > Points for Indoors: By using planes as the fundamental building block, the model naturally enforces the structural constraints of indoor scenes, leading to cleaner reconstructions with fewer primitives.
- Bypassing the "Detection Bottleneck": By directly optimizing primitives against monocular cues, PlanarSplatting avoids the errors that typically accumulate in traditional "detect-then-match" pipelines.
- Synergy with Gaussian Splatting (GS): One of the most exciting findings is that these reconstructed planes provide an excellent geometric "skeleton" for GS. Initializing 3DGS with these planes leads to better rendering quality in less time compared to standard point-based initialization.
Key Results
PlanarSplatting achieves a remarkable balance of speed and accuracy:
- Speed: It reconstructs a complete scene in approximately 3 minutes on a single GPU.
- Geometric Accuracy: On ScanNetV2, it achieved a Chamfer distance of 4.83, outperforming prior methods like PlanarRecon (9.89) and AirPlanes (5.30).
- Novel View Synthesis: When integrated with 3D Gaussian Splatting (3DGS), it improved PSNR from 24.4 to 25.5 while using 70% fewer points and cutting optimization time by more than half.
Qualitatively, the method captures fine details—like chair legs and bed frames—that previous planar methods often smoothed over (see Figure 6).
Limitations and Future Work
The primary limitation is the planar assumption. While excellent for walls and floors, the current method is not suited for complex, curved shapes (e.g., a round vase or a plush sofa). The authors suggest that future work could involve developing more flexible geometric primitives to handle these non-planar elements while maintaining the efficiency of the splatting framework.
Summary updated