GAPS: Geometry-Aware, Physics-Based, Self-Supervised Neural Garment Draping

Ruochen Chen, Liming Chen, Shaifali Parashar

TL;DR

This paper introduces GAPS, a novel neural network for garment draping that combines physics-based simulation with geometry-aware constraints. GAPS achieves more realistic and collision-free garment simulations by enforcing inextensibility where possible and intelligently allowing stretching only over larger body regions, outperforming existing methods, especially on loose garments.

What Problem Does This Paper Address?

Simulating how clothes drape on a body is crucial for applications like virtual try-on, fashion design, and video games. Traditional physics-based simulations are accurate but computationally expensive, making them unsuitable for real-time use. Newer neural network approaches offer speed but often struggle with realistic garment behavior. Specifically, existing methods tend to either:

  • Produce physically implausible stretching: They over-stretch garments everywhere to avoid collisions, leading to unrealistic results.
  • Require expensive post-processing: Collisions are fixed after the initial simulation, which can introduce further stretching artifacts.
  • Lack scalability: Some methods require retraining for each new body shape or struggle with loose-fitting garments (like dresses) due to flawed "skinning" (how the garment attaches to the body's skeleton).

This paper aims to address these limitations by creating a faster, more robust, and visually accurate garment draping system that handles collisions and garment types more effectively.

Methods

GAPS builds upon existing physics-based simulation principles and neural network architectures but introduces two key innovations:

  • Geometry-Aware Inextensibility Loss: Instead of uniformly enforcing inextensibility, GAPS uses a novel loss function that preserves local distances and areas. This constraint is only gradually relaxed when a collision is detected, allowing the garment to stretch naturally over larger body parts while preventing penetration. This is achieved by analyzing the covariance matrix of local mesh neighborhoods.
  • Geometry-Aware Skinning: Traditional skinning methods often rely on simple nearest-neighbor body vertex assignments, which fail for loose garments. GAPS introduces a new skinning approach that uses a Gaussian Radial Basis Function (RBF) to compute body-participation weights. This method considers the distance between garment vertices and body vertices, allowing it to automatically find the most relevant body region for motion transfer, thus handling both tight and loose garments more effectively.

The overall model is trained using a composite loss function that includes standard physics-based losses (strain, gravity, collision, bending, inertia) and the new inextensibility loss.

Datasets

The paper uses the AMASS dataset for training and evaluation. Specifically, they train on 60 sequences and evaluate on 5 validation sequences from AMASS, containing approximately 11,000 poses. They also mention using sequences from AMASS 86_07 for detailed evaluation.

Key Insights and Ideas

  • Inextensibility is not absolute: Realistic garments only stretch where necessary to accommodate body shape. Forcing inextensibility everywhere leads to unnatural results, while allowing it everywhere leads to collisions. GAPS finds a smart middle ground.
  • Geometry is key to skinning: The way a garment attaches to the body's skeleton (skinning) is critical for realism, especially for loose clothing. Simple nearest-neighbor approaches are insufficient; a more nuanced, geometry-aware method is needed.
  • Self-supervised learning is powerful: By leveraging physics-based constraints as loss functions, the model can learn realistic draping without requiring explicit ground truth 3D scans of draped garments, which are expensive and difficult to obtain.

Key Results

  • Improved Realism: GAPS produces visually more realistic drapings, especially for loose garments like dresses, closely matching traditional physics-based simulation results.
  • Reduced Collisions: The collision-aware inextensibility loss significantly reduces body-garment penetration (ɛc metric) compared to other methods. For example, on a dress, GAPS achieves an ɛc of 0.196, while SNUG has 3.113 and NCS has 0.368.
  • Minimized Stretching: GAPS also shows lower stretching (ɛe and ɛa metrics) compared to methods that rely heavily on post-processing to fix collisions.
  • Scalability: GAPS is trained on variable bodies and demonstrates good performance across diverse body shapes without requiring retraining for each.
  • Efficiency: While training can take up to 8 hours for looser garments on 8 NVIDIA V100 GPUs, the inference time is significantly faster than SNUG, with GAPS achieving 2.7 ms per frame compared to SNUG's 23.7 ms.

Limitations and Future Work

  • Cloth Dynamics: The authors acknowledge that while temporal neural networks are incorporated, there is still room for improvement in handling complex cloth dynamics.
  • Extreme Poses/Shapes: Despite mitigating collision issues, challenges may persist with extreme body poses and shapes.
  • Self-Collision and Multi-Layering: The current framework does not address cloth self-collision or multi-layer clothing.
  • Future Directions: The authors suggest exploring the imposition of their collision-aware inextensibility as a hard constraint for guaranteed zero collisions and investigating solutions for self-collision and multi-layer garments.

Summary updated

Abstract

Recent neural, physics-based modeling of garment deformations allows faster and visually aesthetic results as opposed to the existing methods. Material-specific parameters are used by the formulation to control the garment inextensibility. This delivers unrealistic results with physically implausible stretching. Oftentimes, the draped garment is pushed inside the body which is either corrected by an expensive post-processing, thus adding to further inconsistent stretching; or by deploying a separate training regime for each body type, restricting its scalability. Additionally, the flawed skinning process deployed by existing methods produces incorrect results on loose garments. In this paper, we introduce a geometrical constraint to the existing formulation that is collision-aware and imposes garment inextensibility wherever possible. Thus, we obtain realistic results where draped clothes stretch only while covering bigger body regions. Furthermore, we propose a geometry-aware garment skinning method by defining a body-garment closeness measure which works for all garment types, especially the loose ones. Our code is publicly available at https://github.com/Simonhfls/GAPS.
BibTeX
@misc{chen2023gaps,
  author = {Ruochen Chen and Liming Chen and Shaifali Parashar},
  title = {{GAPS: Geometry-Aware, Physics-Based, Self-Supervised Neural Garment Draping}},
  year = {2023},
  eprint = {2312.01490},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.01490},
}
Figure