2DGS-Avatar: Animatable High-fidelity Clothed Avatar via 2D Gaussian Splatting

Qipeng Yan, Mingyang Sun, Lihua ZhangarXiv 2025 · OpenReview

TL;DR

This paper introduces 2DGS-Avatar, a novel method for creating high-fidelity, animatable clothed human avatars from monocular videos. By leveraging 2D Gaussian Splatting (2DGS), it achieves fast training and real-time rendering while improving geometric accuracy over existing 3D Gaussian Splatting methods.

What Problem Does This Paper Address?

Creating realistic, animated human avatars from video is a long-standing challenge in computer graphics and vision. While methods like Neural Radiance Fields (NeRF) offer impressive visual quality, they are computationally expensive and slow to render. More recent approaches using 3D Gaussian Splatting (3DGS) have significantly improved rendering speed but often struggle with accurate geometry, leading to artifacts, especially for thin surfaces like clothing. This paper aims to bridge this gap by developing a method that combines the speed of Gaussian Splatting with improved geometric fidelity for animatable avatars.

Methods

The core of 2DGS-Avatar lies in its adaptation of 2D Gaussian Splatting (2DGS) for human avatars. Here's a breakdown of their approach:

  • Canonical Space Representation: The avatar is initially represented by 2D Gaussian primitives in a "canonical space." This space is linked to the SMPL-X model, a standard parameterized human model.
  • Forward Skinning: To animate the avatar, these 2D Gaussian primitives are transformed from the canonical space to a "posed space" using Linear Blend Skinning (LBS). This process leverages precomputed skinning weights diffused from the SMPL-X surface.
  • 2D Gaussian Splatting (2DGS): Instead of 3D ellipsoids used in 3DGS, 2DGS employs 2D ellipses (surfels) to represent the avatar's surface. This is inspired by the observation that 2DGS can converge to more accurate geometry than 3DGS.
  • Differentiable Rasterization: The method uses a differentiable rasterizer to render images and depth maps from the posed 2D Gaussians.
  • Optimization Losses: The model is trained using a combination of losses:
  • Photometric Loss: Minimizes the difference between rendered and input RGB images, enhanced with LPIPS for perceptual similarity.
  • Normal Loss: Uses surface normals estimated from rendered depth maps and ground truth normals (derived from input RGB images via PIFuHD) to enforce geometric accuracy.
  • Self-supervised Loss: Encourages uniform distribution of Gaussian primitives on the surface and pushes their opacity towards 0 or 1, improving surface adherence.
  • Mask Loss: Ensures the rendered avatar aligns with the input mask.
  • Eccentricity Filtering: A novel technique to refine the adaptive density control of 2DGS. It removes overly elongated Gaussian ellipses, leading to smoother geometric edges.

Datasets

The paper evaluates their method on two popular datasets:

  • AvatarRex [24]: Contains sequences of clothed human avatars.
  • THuman4.0 [27]: Another dataset for human reconstruction and animation.

They also mention using PIFuHD [23] to generate normal maps from RGB images for supervision.

Key Insights and Ideas

  • 2DGS for Avatars is a Game Changer: The core insight is that adapting 2D Gaussian Splatting, which inherently favors more accurate geometry than its 3D counterpart, is a promising direction for animatable avatar reconstruction.
  • Geometry Matters for Realism: The paper strongly emphasizes the importance of geometric accuracy, arguing that photometric loss alone is insufficient. The inclusion of a normal loss is crucial for achieving high-fidelity results.
  • Self-Supervision for Surface Adherence: The proposed self-supervised loss is key to ensuring that the Gaussian primitives are not just present but are well-distributed and adhere tightly to the avatar's surface, preventing floating artifacts.
  • Eccentricity Filtering for Smoother Edges: This novel filtering mechanism addresses a subtle but important issue in Gaussian splatting – the potential for elongated primitives to create jagged edges.

Key Results

  • High-Fidelity Rendering: 2DGS-Avatar produces visually superior rendering quality compared to GauHuman [11], with clearer textures on clothing and facial features.
  • Fast Training: The method achieves training times of around 1 hour, comparable to GauHuman [11] and significantly faster than Animatable Gaussians [10] (7 hours).
  • Real-time Rendering: The reconstructed avatars can be rendered at up to 60 FPS on a single NVIDIA RTX-4070s GPU, making them suitable for real-time applications.
  • Quantitative Performance: While slightly behind Animatable Gaussians [10] in some metrics, 2DGS-Avatar achieves comparable results in a fraction of the training time and with less GPU memory. For instance, on AvatarRex, the full model achieves a PSNR of 31.36, SSIM of 0.9781, and LPIPS of 34.75.

Limitations and Future Work

  • Input Blurriness: The paper acknowledges that blurry input RGB videos can degrade reconstruction quality, especially in shadowed areas.
  • Clothing Wrinkle Realism: While the avatars are high-fidelity, simulating highly realistic clothing wrinkles remains a challenge. The authors point to IF-Garments [31] as a potential direction for modeling such details.
  • Less Observed Regions: Modeling regions that are less frequently observed in training data (e.g., underarms, shoe soles) is still an area for improvement.
  • Future Directions: The authors suggest that further work could focus on improving clothing wrinkle simulation and handling less frequently observed body parts.

Summary updated

Abstract

Real-time rendering of high-fidelity and animatable avatars from monocular videos remains a challenging problem in computer vision and graphics. Over the past few years, the Neural Radiance Field (NeRF) has made significant progress in rendering quality but behaves poorly in run-time performance due to the low efficiency of volumetric rendering. Recently, methods based on 3D Gaussian Splatting (3DGS) have shown great potential in fast training and real-time rendering. However, they still suffer from artifacts caused by inaccurate geometry. To address these problems, we propose 2DGS-Avatar, a novel approach based on 2D Gaussian Splatting (2DGS) for modeling animatable clothed avatars with high-fidelity and fast training performance. Given monocular RGB videos as input, our method generates an avatar that can be driven by poses and rendered in real-time. Compared to 3DGS-based methods, our 2DGS-Avatar retains the advantages of fast training and rendering while also capturing detailed, dynamic, and photo-realistic appearances. We conduct abundant experiments on popular datasets such as AvatarRex and THuman4.0, demonstrating impressive performance in both qualitative and quantitative metrics.
BibTeX
@misc{yan2025dgs,
  author = {Qipeng Yan and Mingyang Sun and Lihua Zhang},
  title = {{2DGS-Avatar: Animatable High-fidelity Clothed Avatar via 2D Gaussian Splatting}},
  year = {2025},
  eprint = {2503.02452},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.02452},
}
Figure