A Neural-Network-Based Approach for Loose-Fitting Clothing

Yongxu Jin, Dalton Omens, Zhenglin Geng, Joseph Teran, Abishek Kumar, Kenji Tashiro, Ronald FedkiwarXiv 2024 · DOI · arXiv

Abstract

Simulating the dynamic behavior of loose-fitting garments like capes and skirts in real-time presents a significant challenge in computer graphics. Traditional physics-based simulations are often too computationally expensive for interactive applications, while pure machine learning approaches frequently fail to capture complex temporal dynamics without excessive training data. This paper presents a hybrid framework that combines a low-degree-of-freedom "rope chain" physics proxy with a multi-stage neural network pipeline. By separating large-scale ballistic motion from high-frequency geometric detail, the proposed method achieves realistic, high-resolution cloth dynamics at real-time speeds.

Introduction

Loose-fitting clothing is characterized by complex dynamic modes, such as billowing and swirling, which are difficult to simulate efficiently. Existing solutions typically face a trade-off between speed, physical accuracy, and detail. High-fidelity physics engines used in film production are far too slow for real-time games or VR. Conversely, neural network models, while capable of approximating static shapes, often struggle with the "ballistic" swinging motion of fabric over time. Recurrent Neural Networks (RNNs) used for this purpose require massive datasets and are prone to numerical drift.

Comparison of the rope chain proxy, the proposed hybrid inference, and full ground-truth simulation for capes and skirts.

Methodology

The core of the proposed approach is a three-stage pipeline that decouples the motion of the garment from its high-resolution shape.

1. Rope Chain Physics Proxy

Instead of a dense mesh, the system simulates a sparse set of vertical "rope chains" that act as virtual bones. These chains are inextensible but can buckle freely, capturing the primary ballistic movement of the garment with 80–90x fewer degrees of freedom than a full simulation.

2. History-Based Collision Handling

To prevent the cloth from clipping through the body mesh—a common issue in low-resolution simulations—the authors implement a history-based collision method. By using the previous frame's non-penetrating position to determine the projection direction, the system significantly improves robustness on thin or high-curvature body parts.

3. Neural Network Stack

The coarse rope chain positions are processed by two lightweight Multi-Layer Perceptrons (MLPs):

  • Neural Skinning: This network maps the proxy positions to the Principal Component Analysis (PCA) coefficients of a coarse mesh.
  • Neural Shape Inference (QNN): A Quasistatic Neural Network adds fine-grained details like wrinkles and folds. Because the physics proxy handles temporal dynamics, these networks do not require memory units (like LSTMs), leading to faster inference and easier training.

Results and Discussion

The model was trained on approximately 5,000 frames of high-fidelity simulation data. Evaluation on both seen and unseen motions demonstrated several key advantages:

  • Efficiency: The reduction in degrees of freedom allows for real-time performance on standard hardware.
  • Generalization: Unlike pure ML models that may fail on out-of-distribution motions, the hybrid approach remains stable because the underlying physics proxy provides a plausible motion baseline.
  • Visual Quality: The results are comparable to full-scale simulations, avoiding the "rubbery" overstretching artifacts common in other real-time solvers.

Conclusion

By leveraging physics for motion and machine learning for detail, this hybrid framework provides a robust solution for real-time loose clothing. Future work includes making the rope chain simulation differentiable to automatically learn material parameters and improving collision handling for extreme out-of-distribution poses.

Summary updated

Abstract

Since loose-fitting clothing contains dynamic modes that have proven to be difficult to predict via neural networks, we first illustrate how to coarsely approximate these modes with a real-time numerical algorithm specifically designed to mimic the most important ballistic features of a classical numerical simulation. Although there is some flexibility in the choice of the numerical algorithm used as a proxy for full simulation, it is essential that the stability and accuracy be independent from any time step restriction or similar requirements in order to facilitate real-time performance. In order to reduce the number of degrees of freedom that require approximations to their dynamics, we simulate rigid frames and use skinning to reconstruct a rough approximation to a desirable mesh; as one might expect, neural-network-based skinning seems to perform better than linear blend skinning in this scenario. Improved high frequency deformations are subsequently added to the skinned mesh via a quasistatic neural network (QNN). In contrast to recurrent neural networks that require a plethora of training data in order to adequately generalize to new examples, QNNs perform well with significantly less training data.
BibTeX
@misc{jin2024neural,
  author = {Yongxu Jin and Dalton Omens and Zhenglin Geng and Joseph Teran and Abishek Kumar and Kenji Tashiro and Ronald Fedkiw},
  title = {{A Neural-Network-Based Approach for Loose-Fitting Clothing}},
  year = {2024},
  eprint = {2404.16896},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.16896},
}
Figure