Towards Multi-Layered 3D Garments Animation

Yidi Shao, Chen Change Loy, Bo Dai

Abstract

Realistic 3D garment animation is a cornerstone of virtual reality, gaming, and digital fashion. However, existing data-driven models often struggle with multi-layered clothing and external environmental forces. This paper introduces LayersNet, a Transformer-based simulation framework designed to animate complex, multi-layered outfits by treating them as a hierarchy of interacting patches. By incorporating a novel Rotation Equivalent Transformation (RET), the framework achieves state-of-the-art realism and efficiency, significantly reducing interpenetration and improving accuracy in dynamic scenarios involving wind and human motion.

Introduction

In the pursuit of immersive digital environments, the realistic simulation of clothing remains a significant challenge. While single-layer garment animation has seen substantial progress, real-world outfits are rarely composed of a single piece of fabric. Most existing models fail to account for the complex friction, pressure, and collisions that occur when multiple layers—such as a jacket over a dress—interact. Furthermore, these models often ignore external forces like wind, which are essential for believable fabric behavior. Modeling these "micro-physics" interactions is computationally expensive and mathematically difficult for standard neural networks to learn from scratch.

The LayersNet Framework

LayersNet shifts the focus from global garment shapes to local particle interactions through three key technical innovations:

1. Two-Level Structural Hierarchy

To avoid the computational burden of simulating every vertex in a high-resolution mesh, LayersNet groups vertices into patches. The model simulates interactions at this patch level, significantly reducing complexity while maintaining high-fidelity detail. This hierarchical approach allows the system to scale to complex outfits without a linear increase in processing requirements.

2. Rotation Equivalent Transformation (RET)

The core mathematical contribution of this work is the Rotation Equivalent Transformation (RET). Physical laws are inherently rotation-invariant; for instance, gravity and wind affect fabric based on relative orientation rather than absolute global coordinates. RET maps high-dimensional features into a "local canonical space." This allows the model to understand the semantics of forces like wind without requiring training data for every possible orientation, leading to much better generalization to unseen poses.

3. Transformer-Based Simulator

The architecture utilizes a Transformer to facilitate "communication" between garment patches, the human body, and the wind field. By treating garment animation as a communication problem between particles, the model effectively resolves collisions and stretching by predicting the next state of vertices based on current multi-layered interactions.

The D-LAYERS Dataset

Recognizing a lack of high-quality data for multi-layered animation, the researchers developed D-LAYERS. This massive dataset contains 700,000 frames of dynamics, featuring 4,900 different combinations of multi-layered garments. The dataset is driven by human motion from the AMASS dataset and incorporates randomly sampled wind strengths and directions, providing a robust foundation for training physics-aware neural simulators.

Experimental Results

LayersNet significantly outperforms existing state-of-the-art models such as DeePSD and MGNet across several metrics:

  • Accuracy: In layered garment tests, LayersNet achieved Euclidean errors (the distance between predicted and ground-truth vertex positions) that were 50-60% lower than baseline models.
  • Collision Reduction: The RET module proved exceptionally effective at reducing "interpenetration." It decreased garment-to-human penetration by over 71% (from 9.51% to 2.72%) compared to versions of the model without the module.
  • Qualitative Realism: The model successfully captures complex inertia effects, such as a jacket sliding off a shoulder during movement or fabric billowing realistically in high-velocity wind.

Conclusion and Future Work

LayersNet represents a significant step toward realistic, efficient multi-layered garment animation. By exploiting physical symmetries through RET and utilizing a patch-based Transformer architecture, the framework handles complex interactions that were previously computationally prohibitive. While the current model is trained on specific garment templates, future work will focus on optimizing the hierarchy for real-time performance on consumer hardware and expanding the system to handle arbitrary topologies, such as capes or scarves, and a wider variety of fabric materials.

Summary updated

Abstract

Mimicking realistic dynamics in 3D garment animations is a challenging task due to the complex nature of multi-layered garments and the variety of outer forces involved. Existing approaches mostly focus on single-layered garments driven by only human bodies and struggle to handle general scenarios. In this paper, we propose a novel data-driven method, called LayersNet, to model garment-level animations as particle-wise interactions in a micro physics system. We improve simulation efficiency by representing garments as patch-level particles in a two-level structural hierarchy. Moreover, we introduce a novel Rotation Equivalent Transformation that leverages the rotation invariance and additivity of physics systems to better model outer forces. To verify the effectiveness of our approach and bridge the gap between experimental environments and real-world scenarios, we introduce a new challenging dataset, D-LAYERS, containing 700K frames of dynamics of 4,900 different combinations of multi-layered garments driven by both human bodies and randomly sampled wind. Our experiments show that LayersNet achieves superior performance both quantitatively and qualitatively. We will make the dataset and code publicly available at https://mmlab-ntu.github.io/project/layersnet/index.html .
BibTeX
@inproceedings{shao2023towards,
  author = {Yidi Shao and Chen Change Loy and Bo Dai},
  title = {{Towards Multi-Layered 3D Garments Animation}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
}
Figure