TL;DR
This paper introduces Superquadric Splats (SQS), a novel 3D representation that extends Gaussian Splatting by using superquadrics as volumetric primitives. This allows for more expressive modeling of complex geometry, leading to significant improvements in both visual and geometric reconstruction accuracy compared to existing methods.
What Problem Does This Paper Address?
Novel View Synthesis (NVS) and 3D reconstruction from multi-view images are crucial for applications in robotics and spatial computing. While recent methods like 3D Gaussian Splatting (3DGS) have achieved impressive real-time rendering and visual fidelity, they struggle with modeling intricate geometry, particularly sharp edges and corners. This is because the underlying primitive, an ellipsoid-shaped Gaussian, has limited expressive power. Approximating complex shapes like boxes often requires a large number of Gaussians, still leading to inaccuracies. This paper aims to address this limitation by proposing a more geometrically expressive primitive.
Methods
The core innovation is the introduction of Superquadric Splats (SQS). Instead of using Gaussian primitives, SQS utilizes superquadrics, a parametric family of shapes that can naturally represent a wider range of geometries, including cubes, cylinders, and spheres, with fewer parameters. The paper treats superquadrics as volumetric distance functions rather than just level-set surfaces.
A key technical challenge was developing a differentiable rendering pipeline to support these superquadric projections. This involved:
- Superquadric Distance Function: Extending the standard superquadric distance function with exponential terms to better capture density variations and high-contrast edges.
- Rim Projection: A novel technique to recover 3D information for each pixel by leveraging the "rim" of the shape (the boundary between visible and occluded sections). This involves projecting the 3D superquadric onto the 2D image plane and using analytic constraints to infer depth.
- Coordinate Systems: Careful management of different coordinate systems (Shape, World, Camera, Image, Pixel) is essential for accurate transformations and rendering.
The SQS approach is designed to be a plug-and-play component for existing 3DGS pipelines, leveraging the same self-supervised training framework.
Datasets
The paper evaluates SQS on both synthetic and real-world datasets:
- Synthetic Datasets: Chair [12] and NeRDS360 [17]. These are used to quantitatively assess geometric reconstruction accuracy due to the availability of ground truth depth maps.
- Real-world Datasets: 12 scenes from Deep Blending [13], 6 scenes from Tanks&Temples [23], and 8 scenes from the MipNeRF360 dataset [3]. For these, qualitative comparisons are presented as ground truth depth is not available.
Key Insights and Ideas
- Superquadrics are inherently more expressive for geometry: The paper strongly demonstrates that a single superquadric can represent shapes like squares or polygons much more accurately than multiple Gaussians, highlighting the fundamental limitations of the Gaussian primitive for complex geometry.
- Volumetric superquadrics, not just surfaces: By treating superquadrics as volumetric distance functions and integrating them into a splatting framework, the method gains the benefits of both efficient rendering (avoiding empty space) and rich geometric representation.
- Rim projection is a clever way to infer 3D from 2D: The rim projection technique provides a crucial analytic constraint to recover 3D information from 2D pixel data, enabling the rendering of superquadric splats.
- SQS is a direct upgrade for 3DGS: The proposed method seamlessly integrates into existing 3DGS pipelines, offering improved geometric modeling with only a small increase in parameters per primitive.
Key Results
- Geometric Reconstruction: SQS consistently achieves a 5-10% improvement in depth reconstruction accuracy (measured by metrics like relative error, log10 error, and RMS error) compared to 3DGS and GES, particularly at edges and sharp corners.
- Visual Reconstruction: SQS shows an average improvement of ~1dB in PSNR over classic Gaussian Splatting.
- Qualitative Improvements: Visualizations show SQS reconstructing finer details like zebra crossings, letter boundaries, and textures more accurately than 3DGS. Figure 2 vividly illustrates how a single SQS can perfectly match a target square, while a single or even ten Gaussians struggle.
Limitations and Future Work
- Analytic Solution for Rim Projection: The paper acknowledges the lack of a closed-form analytic solution for the superquadric rim projection ($f(\theta) = 0$). Currently, this is addressed by storing and searching through sampled rim values, which is computationally less efficient.
- Future Work: The authors suggest developing a better analytic relation for the rim projection to reduce rendering costs. They also mention that a CUDA implementation would significantly improve rendering speed. The method is particularly well-suited for man-made structures with sharp edges and corners, and future work could explore its performance on more textured or organic scenes.
Summary updated