Abstract
Digitalizing clothed humans from sparse RGB cameras remains a challenge due to the trade-off between reconstruction speed and visual fidelity. We present GBC-Splat, a feed-forward pipeline that reconstructs high-quality, animatable 3D humans in under 0.3 seconds. By anchoring 2D Gaussian surfels to a refined mesh and utilizing a novel subdivision algorithm, GBC-Splat achieves state-of-the-art rendering quality and geometry without requiring time-consuming per-subject optimization or relying on restrictive parametric human models.
Introduction
Creating high-fidelity digital twins of clothed humans typically necessitates either dense camera arrays or extensive per-subject optimization. While Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3D-GS) have advanced the field, they often struggle with sparse data or require minutes of training for each new pose. Furthermore, many generalizable methods rely on human parametric models like SMPL, which are inadequate for loose clothing such as dresses or capes. GBC-Splat addresses these limitations by providing an instant, high-fidelity, and generalizable digitalization framework capable of handling complex geometry from only a few camera views.
Methodology
The GBC-Splat pipeline consists of three primary feed-forward stages designed to extract geometry and appearance efficiently.
1. Coarse-to-fine Mesh Extraction
The process begins with a dual-path geometric reconstruction. A U-Net-like encoder extracts 2D and 3D features, which are fused via cross-attention to regress an implicit occupancy field for coarse topology. Simultaneously, explicit stereo-matching (disparity estimation) is performed between camera pairs to capture high-frequency surface details like wrinkles. These representations are fused using Poisson Surface Reconstruction to produce a watertight, detailed mesh.
2. Mesh-anchored 2D Gaussian Primitives
To represent appearance, GBC-Splat utilizes 2D Gaussian surfels instead of 3D volumes. Each Gaussian is anchored to a vertex on the extracted mesh, with its normal vector locked to the mesh surface. An MLP predicts properties such as scale, opacity, and color based on fused image features, ensuring the appearance is strictly tied to the underlying geometry.
3. Subdivision-based Upsampling
To overcome the spectral bias of neural networks toward low-frequency information, we introduce a texel-based subdivision module. During inference, Gaussians covering areas with high color variance are split into four smaller primitives. This densification occurs without additional training, significantly enhancing the sharpness of textures and fine details.
Experimental Results
GBC-Splat was evaluated on the Twindom, THuman2.0, and 2K2K datasets. The pipeline demonstrates exceptional efficiency, completing reconstruction in less than 0.3 seconds.
In terms of geometry, GBC-Splat achieved a Chamfer Distance of 1.218, significantly outperforming NeuS2 (~6.3) and DoubleField (~5.0) in sparse settings. For rendering quality, the model achieved a PSNR of 25.86 and an LPIPS of 0.1188 on THuman2.0 and Twindom, surpassing existing generalizable methods like GPS-Gaussian and GHG. Qualitative results highlight cleaner edges and more realistic clothing folds, particularly in cases involving non-standard human shapes.
Conclusion
GBC-Splat provides a robust solution for real-time human digitalization from sparse views. By combining implicit and explicit geometric representations with anchored Gaussian splatting and subdivision-based upsampling, it achieves high-fidelity results without per-subject optimization. Future work will focus on improving temporal consistency to reduce flickering in volumetric video applications.
Summary updated