SMPLX-Lite: A Realistic and Drivable Avatar Benchmark with Rich Geometry and Texture Annotations

Yujiao Jiang, Qingmin Liao, Zhaolong Wang, Xiangru Lin, Zongqing Lu, Yuxi Zhao, Hanqing Wei, Jingrui Ye, Yu Zhang, Zhijing Shao

Introduction

Creating photorealistic, drivable 3D human avatars is a cornerstone objective for the future of virtual reality, gaming, and telepresence. However, a significant "representation gap" persists between standard parametric models and the complexity of real-world clothed humans. While models like SMPL and SMPL-X provide robust frameworks for body pose and shape, they are fundamentally designed for "naked" bodies. Attempts to adapt these models to clothed subjects often result in unnatural mesh distortions, such as "lip flipping" or eye-socket artifacts, particularly when attempting to capture high-frequency details like clothing wrinkles.

Furthermore, existing datasets often lack the necessary synergy between high-resolution imagery and unified 3D topology. Researchers are frequently forced to choose between high-quality images without underlying 3D models or 3D scans that lack the temporal consistency required for animation. This paper introduces SMPLX-Lite, a comprehensive benchmark and dataset designed to bridge these gaps through a refined parametric model, a robust fitting pipeline, and a massive high-resolution data repository.

Methodology

The SMPLX-Lite-D Model

The authors propose SMPLX-Lite-D, a streamlined version of the industry-standard SMPL-X model. By strategically removing internal vertices—such as those representing the eyeballs and cochlea—and flattening specific regions like toe seams, the vertex count was reduced from 10,475 to 8,452. This simplification is not merely for efficiency; it significantly improves stability during "vertex fitting." By reducing the complexity of the mesh in areas prone to self-intersection, the model avoids the common geometric artifacts that plague more complex topologies when they are deformed to match clothed scans.

Multi-Stage Fitting and Generation

To capture the intricate dynamics of clothing, the authors employ a two-stage fitting process:

  1. Embedded Nodes: This stage warps the general shape of the SMPLX-Lite-D mesh to match the coarse volume of the subject.
  2. Vertex Shifts: This stage solves for fine-grained displacements to capture high-frequency geometric details, such as folds and wrinkles in fabric.

To demonstrate the utility of this representation, the authors developed a Conditional Variational Autoencoder (CVAE). This generative model takes human pose and facial keypoints as input to produce photorealistic, textured 3D avatars that can be driven by novel motion sequences.

The SMPLX-Lite Dataset

The core contribution of this work is the SMPLX-Lite dataset, which provides a new standard for avatar research. The dataset features five individuals (three male, two female) wearing daily attire and performing 15 distinct action sequences, ranging from debating to stretching.

The data was captured using a sophisticated rig consisting of 32 calibrated 4K RGB cameras and 48 infrared cameras for depth. Notably, the inclusion of eight telephoto lenses allows for the capture of "local details"—such as finger joints and facial micro-expressions—that are often lost in standard wide-angle setups. The resulting dataset includes over 20,000 frames of multi-view segmented images, 3D keypoints, textured scanned meshes, and fitted SMPLX-Lite-D models with corresponding UV-space texture maps.

Experimental Results

Quantitative Performance

SMPLX-Lite was evaluated against state-of-the-art baselines, including Neural Body and Ani-NeRF. In tasks involving Novel View Synthesis and Novel Pose Synthesis, the proposed method consistently outperformed competitors. For instance, on the subject "WZL," the model achieved a PSNR of 26.54 and an SSIM of 0.9443, representing a significant margin over existing methods.

Qualitative and Geometric Accuracy

Qualitatively, the generated avatars exhibit superior texture clarity and geometric integrity. While baseline models often struggle with "blurry" hands or "twisted" limbs during extreme poses, SMPLX-Lite maintains sharp details and accurate finger movements. The geometric fidelity is further evidenced by a low Chamfer Distance (ranging from $6.4$ to $8.2 \times 10^{-3}$), indicating that the fitted models closely mirror the original high-resolution scans.

Discussion and Future Work

While SMPLX-Lite represents a significant advancement, certain challenges remain. Raw scans can still suffer from "mesh adhesion," where limbs in close proximity (e.g., a hand touching a thigh) are incorrectly fused. Additionally, like all data-driven approaches, the model may produce artifacts when faced with extreme poses that fall outside the training distribution.

Future research will focus on improving facial control. The current driving algorithm for expressions is relatively elementary; decoupling facial expressions from body poses will be essential for creating more vivid, emotionally expressive avatars. Nevertheless, SMPLX-Lite provides the community with a powerful new foundation for the next generation of digital humans.

Summary updated

Abstract

Recovering photorealistic and drivable full-body avatars is crucial for numerous applications, including virtual reality, 3D games, and tele-presence. Most methods, whether reconstruction or generation, require large numbers of human motion sequences and corresponding textured meshes. To easily learn a drivable avatar, a reasonable parametric body model with unified topology is paramount. However, existing human body datasets either have images or textured models and lack parametric models which fit clothes well. We propose a new parametric model SMPLX-Lite-D, which can fit detailed geometry of the scanned mesh while maintaining stable geometry in the face, hand and foot regions. We present SMPLX-Lite dataset, the most comprehensive clothing avatar dataset with multi-view RGB sequences, keypoints annotations, textured scanned meshes, and textured SMPLX-Lite-D models. With the SMPLX-Lite dataset, we train a conditional variational autoencoder model that takes human pose and facial keypoints as input, and generates a photorealistic drivable human avatar.
BibTeX
@inproceedings{jiang2024smplx,
  author = {Yujiao Jiang and Qingmin Liao and Zhaolong Wang and Xiangru Lin and Zongqing Lu and Yuxi Zhao and Hanqing Wei and Jingrui Ye and Yu Zhang and Zhijing Shao},
  title = {{SMPLX-Lite: A Realistic and Drivable Avatar Benchmark with Rich Geometry and Texture Annotations}},
  booktitle = {ICME},
  year = {2024},
  doi = {10.1109/icme57554.2024.10687388},
}
Figure