MBDS: A Multi-Body Dynamics Simulation Dataset for Graph Networks Simulators

Sheng Yang, Fengge Wu, Junsuo ZhaoarXiv 2024 · DOI · arXiv

Researchers have introduced MBDS (Multi-Body Dynamics Simulation), a massive new dataset designed to push Graph Network Simulators (GNS) beyond simple particle interactions. Featuring 150,000 trajectories across eight complex 1D, 2D, and 3D mechanical scenarios, MBDS provides a rigorous benchmark that reveals how current "physics-informed" models often struggle when faced with real-world engineering complexities like friction, rotation, and external forces.

The Challenge of Real-World Dynamics

Graph Network Simulators (GNS) have become the go-to method for modeling physical phenomena because they are computationally efficient and highly accurate. However, most existing datasets used to train these models are "toy" examples—think of a few spheres colliding or a simple fluid flow.

In the real world, industrial and engineering problems involve multi-body dynamics: complex mechanical structures where multiple parts are linked, rotating, and experiencing friction simultaneously. Current datasets lack the volume and complexity to test if GNS models can actually handle these "messy" real-world conditions. The authors identify a significant gap: models that perform well on simple pairs of objects often fail when integrated into a complex system like a four-wheeled car or a multi-link rope ladder.

Comparison of conventional simple interactions versus the complex mechanical interplay featured in MBDS.

Methodology and Benchmarking

The authors moved away from error-prone computer vision or sensor-based data collection. Instead, they utilized MuJoCo, a high-precision physics engine, to generate synthetic data that strictly adheres to physical laws while incorporating realistic "noise" (like air resistance and friction). To evaluate the dataset, they benchmarked four prominent GNS architectures: standard GNS, Hamiltonian Graph Neural Networks (HGNN), Lagrangian Graph Neural Networks (LGNN), and Subequivariant Graph Neural Networks (SGNN).

Dataset Overview

The core contribution is the MBDS dataset, which is significantly larger and more diverse than its predecessors like RigidFall or Cub-Toss.

  • Scale: 150,000 trajectories (13,500 for training, 1,500 for evaluation per scenario) with 200 time-steps each.
  • Scenarios: Eight distinct environments including 3D (ParticleCar, SimpleCubli, 3-Ladder, 2-Ladder, 6-Pendulum, 5-Pendulum), 2D (BallDrop), and 1D (BallScrol).
  • Format: Provided in .csv, .h5, and .npz to ensure compatibility and scalability for different research pipelines.
A comparative overview of MBDS against existing physical simulation datasets, highlighting its superior scale and complexity.

Key Insights and Results

The "Physics Prior" Trap

One of the most surprising findings was that "physics-informed" models (HGNN, LGNN, SGNN) often underperformed the basic GNS on complex scenes like the ParticleCar. These models assume specific conservation laws or symmetries that are broken when external forces, such as wind or motor torque, are applied. In the ParticleCar scene at $t=1$, the basic GNS achieved a Mean Squared Error (MSE) of $0.72 \times 10^{-2}$, while the "physics-aware" HGNN lagged behind at $2.89 \times 10^{-2}$.

Exponential Error Growth

The paper highlights that while GNS models are accurate for short-term predictions ($t=1$), their error grows exponentially as the simulation "rolls out" over time ($t=50$ or $100$). This underscores a lack of long-term robustness in current architectures. Additionally, predictive accuracy declines sharply as the speed of objects increases, suggesting that current models struggle to capture high-frequency dynamics.

Future Directions

While MBDS is a major step forward, the authors acknowledge it is just the beginning. Real-world engineering involves thousands of variations, and the authors plan to expand the diversity of categories in future versions. Furthermore, experiments showed that models trained on one scenario often fail when the environment is slightly altered (e.g., changing wind speed), indicating that developing models that can generalize across different physical environments remains a critical gap in the field.

Summary updated

Abstract

Modeling the structure and events of the physical world constitutes a fundamental objective of neural networks. Among the diverse approaches, Graph Network Simulators (GNS) have emerged as the leading method for modeling physical phenomena, owing to their low computational cost and high accuracy. The datasets employed for training and evaluating physical simulation techniques are typically generated by researchers themselves, often resulting in limited data volume and quality. Consequently, this poses challenges in accurately assessing the performance of these methods. In response to this, we have constructed a high-quality physical simulation dataset encompassing 1D, 2D, and 3D scenes, along with more trajectories and time-steps compared to existing datasets. Furthermore, our work distinguishes itself by developing eight complete scenes, significantly enhancing the dataset's comprehensiveness. A key feature of our dataset is the inclusion of precise multi-body dynamics, facilitating a more realistic simulation of the physical world. Utilizing our high-quality dataset, we conducted a systematic evaluation of various existing GNS methods. Our dataset is accessible for download at https://github.com/Sherlocktein/MBDS, offering a valuable resource for researchers to enhance the training and evaluation of their methodologies.
BibTeX
@misc{yang2024mbds,
  author = {Sheng Yang and Fengge Wu and Junsuo Zhao},
  title = {{MBDS: A Multi-Body Dynamics Simulation Dataset for Graph Networks Simulators}},
  year = {2024},
  eprint = {2410.03107},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.03107},
}
Figure