Abstract
LayGA (Layered Gaussian Avatars) is a novel framework that leverages 3D Gaussian Splatting to create animatable digital humans with decoupled layers for body and clothing. By addressing the "entanglement" of geometry in traditional models, LayGA enables high-fidelity virtual try-on and realistic garment animation. The method introduces geometric constraints and a two-stage training pipeline to ensure smooth surfaces and accurate collision handling, outperforming existing single-layer models in both geometric accuracy and rendering quality.
Introduction
Creating realistic digital avatars is a cornerstone of the metaverse, gaming, and e-commerce. While recent advances like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have made photorealistic rendering possible, most models treat the human body and clothing as a single, "entangled" entity.
This entanglement creates two major hurdles:
- Virtual Try-on: Swapping a garment between avatars is difficult when they are baked into the same geometry.
- Motion Fidelity: Real clothes slide over the skin. Entangled models often fail to capture this "sliding" effect, leading to visual artifacts during complex movements.
Furthermore, while 3DGS is excellent for detail, it produces an "unstructured" cloud of points rather than a smooth surface. Without an explicit surface, handling collisions—such as preventing a shirt from clipping through the chest—is extremely challenging.
Methods
The authors propose a two-stage training pipeline to build Layered Gaussian Avatars:
Stage 1: Single-Layer Reconstruction & Segmentation
The model first learns a unified representation of the clothed person. To solve the "unstructured" nature of 3DGS, the authors introduce geometric constraints, including an image-based normal loss and Total Variation (TV) loss. This forces the Gaussians to align into a smooth, continuous surface. Simultaneously, the model learns to segment the Gaussians into "body" and "clothing" labels.
Stage 2: Multi-Layer Fitting
The avatar is split into two distinct models. A key innovation is the separation of Geometry and Rendering Layers. The Geometry Layer is kept smooth for collision handling, while the Rendering Layer adds fine-grained offsets to capture high-frequency details like wrinkles and fabric textures.
Key Innovations
- Gaussian Maps: By parameterizing 3D Gaussians onto a 2D "Gaussian Map" (similar to a UV texture), the authors can treat the point cloud like an image. This allows the use of standard convolutional neural networks (StyleUNet) to predict Gaussian parameters.
- Collision Resolution: At test time, a Laplacian-based post-processing step is used. This allows garments to deform to fit new body shapes while preserving original local structures (like wrinkles), ensuring clothes stay "outside" the skin without full physical simulation.
Results
The researchers evaluated LayGA using the AvatarRex and ActorsHQ datasets, along with custom multi-view sequences.
Geometric and Rendering Quality
Qualitatively, LayGA produces much cleaner point clouds and more accurate surface normals than baselines like Animatable Gaussians. Quantitatively, LayGA achieves superior results across standard metrics. For example, on the ActorsHQ A02 subject, it reached a PSNR of 28.57 (compared to the baseline's 28.25) and improved LPIPS scores, indicating higher perceptual quality.
Clothing Transfer
The model successfully demonstrated "clothing transfer," where a shirt from one subject was realistically draped and animated on four different body types, maintaining realistic fabric details throughout the motion.
Discussion and Limitations
LayGA provides a clever solution for decoupling physics from aesthetics, providing a smooth mathematical surface for collision physics without sacrificing photorealistic detail. However, limitations remain:
- Extreme Body Shapes: Geometric collision handling can still fail if a very tight garment is transferred to a significantly larger body.
- Occlusion: The model cannot reconstruct skin that was occluded during training (e.g., arms under long sleeves).
- Physical Realism: As a purely geometric model, it does not account for gravity or specific material properties (e.g., silk vs. denim).
Future work may involve incorporating physical simulation priors to further enhance the realism of clothing transfer.
Summary updated