TL;DR
Spaghetti Splatting generalizes the popular 3D Gaussian Splatting framework by replacing point-based primitives with convolved polylines. This "physics-ready" representation excels at reconstructing thin, elongated structures like hair and wires while remaining competitive on general scenes, effectively bridging the gap between discrete splats and continuous geometry.
What Problem Does This Paper Address?
In the world of 3D computer vision, we’ve recently seen a massive shift toward 3D Gaussian Splatting (3DGS). It’s fast and produces stunning images, but it has a fundamental limitation: it represents everything as a cloud of disconnected points (Gaussians).
While this works for solid surfaces, it struggles with "curvilinear" geometry—things like hair, spaghetti, or the thin spokes of a bicycle wheel. Because 3DGS uses points, it often represents a single smooth wire as a "string of pearls," which lacks structural continuity. This makes it difficult to use the reconstructed models for anything beyond looking at them, such as physics simulations or traditional animation.
The challenge is that lines and curves are "Dirac masses"—they have zero volume. To make them "learnable" via gradient descent, you need to turn them into a volumetric density that a camera ray can actually hit.
Methods
The authors introduce Spaghetti Splatting, which treats 3D scenes as a collection of learnable polylines. The core "recipe" is a generalization of the one used in 3DGS:
- The Primitive: Instead of a point, they start with a 1D line segment.
- The Convolution: They convolve this segment with a low-frequency kernel (like a Gaussian or a Beta kernel). This "puffs up" the infinitesimal line into a volumetric tube.
- Closed-Form Rendering: To keep things fast, the authors derived a closed-form integral for volume rendering these convolved segments. This allows the system to calculate exactly how much light a "tube" absorbs without expensive numerical sampling.
- Structural Optimization: They use an "attraction loss" to pull the endpoints of nearby segments together and a merging strategy to form long, continuous strands.
Because a point is just a curve with zero length, Spaghetti Splatting is a strict superset of 3D Gaussian Splatting. If the optimization decides a segment should be a point, it can simply shrink its length to zero.
Datasets
The researchers evaluated their method across several diverse benchmarks:
- General Scenes: The standard Mip-NeRF 360 dataset (outdoor and indoor scenes).
- Fibrous/Thin Geometry: The Shelly dataset (synthetic fur and leaves) and the Fuzzy dataset (captured objects like cactus spines and brush bristles).
- Hair Reconstruction: A subset of GeomHair (400 strand-level hairstyles) and the classic Yuksel curly hair model.
Key Insights and Ideas
- The Convergence Property: One of the most elegant findings is that as you subdivide a polyline into smaller segments, the convolved result converges perfectly to the continuous curve. This means the representation is mathematically robust regardless of how many segments you use to define a strand.
- Physics-Ready Output: Unlike point clouds, the output of Spaghetti Splatting is a set of connected polylines. These can be dropped directly into a position-based dynamics (PBD) simulator, allowing for immediate physical interaction with the reconstructed scene.
- Linearization for Speed: To maintain the real-time performance of 3DGS, the authors linearized the camera projection per segment (using EWA splatting). This allows them to handle complex kernels (like Beta kernels) that offer sharper edges than standard Gaussians.
Key Results
- State-of-the-Art in Hair: On hair reconstruction tasks, Spaghetti Splatting outperformed the previous leader, Gaussian Haircut, by 2.7 dB in PSNR and showed a massive 14.3-point improvement in the F1 score for geometric accuracy.
- Competitive on General Scenes: Even on scenes not dominated by lines (like the Mip-NeRF 360 set), it matched or slightly exceeded the performance of state-of-the-art point-based methods like 3DGS-MCMC and Deformable Beta Splatting.
- Efficiency: Despite the more complex primitives, the method maintains high frame rates (e.g., ~177 FPS on Mip-NeRF 360 scenes using an RTX 4090), making it practical for real-time applications.
Limitations and Future Work
While a major step forward, the authors note a few areas for improvement:
- Texture vs. Geometry: Currently, to represent high-frequency color changes along a strand, the system needs many short segments. Integrating learnable texture maps onto the curvilinear primitives could significantly reduce the primitive count.
- Surface Generalization: The authors suggest that by interpolating kernels between endpoints, this family of primitives could be extended to represent cones, triangles, and eventually full surfaces, potentially unifying point, line, and mesh splatting into a single framework.
- Hair Priors: Real-world hair reconstruction is still incredibly difficult; the authors believe integrating learned hair-growth priors could further improve results in extremely occluded areas.
Summary updated