Extending Lagrangian and Hamiltonian Neural Networks with Differentiable Contact Models

Yaofeng Desmond Zhong, Biswadip Dey, Amit Chakraborty

Overview

This paper introduces a differentiable contact model that allows Lagrangian and Hamiltonian Neural Networks to handle real-world physical systems involving collisions and friction. By formulating contact mechanics as a differentiable convex optimization layer, the authors enable neural networks to simultaneously learn continuous system dynamics and discontinuous contact properties—like friction and restitution—directly from trajectory data.

The Problem: Smooth Physics vs. Discontinuous Reality

In recent years, Lagrangian Neural Networks (LNNs) and Hamiltonian Neural Networks (HNNs) have gained popularity for their ability to bake "physics priors" (specifically, energy conservation) directly into neural architectures. However, these models traditionally rely on Ordinary Differential Equations (ODEs), which assume that trajectories are perfectly smooth.

The real world is rarely that smooth. Robots step on the ground, balls bounce, and grippers grab objects. These events create discontinuities: sudden, abrupt changes in velocity. Standard LNNs and HNNs cannot "see" through these impacts. Conversely, existing models that can handle collisions often ignore the underlying energy-conserving ODEs that govern the motion between impacts. This paper bridges that gap, creating a unified framework for learning physics that is both energy-conserving and impact-aware.

Methodology: Differentiable Contact Models

The authors propose a Differentiable Contact Model integrated directly into the computation graph of the neural network. The core approach involves:

  1. Two-Phase Contact Logic: Instead of treating a collision as a single jump, the model splits it into a compression phase (storing energy and stopping) and a restitution phase (releasing energy and bouncing).
  2. Optimization as a Layer: Each phase is formulated as a convex optimization problem.
  3. The compression phase minimizes kinetic energy subject to friction cone constraints.
  4. The restitution phase follows Poisson’s hypothesis, which is more physically robust for complex 3D rotations than the more common Newton’s hypothesis.
  5. Differentiability: To train the network, the authors use CvxpyLayers to differentiate through these optimization problems. This allows gradients to flow from the trajectory error back to the physical parameters, such as the coefficient of friction ($\mu$) and the coefficient of restitution ($e_P$).
  6. Cartesian Coordinates: Unlike many physics engines that use generalized coordinates, this model uses Cartesian coordinates with explicit constraints, which has been shown to improve the learning of system properties.

Evaluated Environments

The authors evaluated their model on eight benchmark tasks across five simulated physical systems, testing various complexities from simple 2D collisions to high-dimensional structures:

Simulated physical systems used for evaluation
  • Bouncing Point Masses (BP5): 2D particles with varying sizes and elasticities.
  • Bouncing Disks (BD5): 2D disks that can rotate, introducing rotational friction.
  • Chained Pendulums (CP3): A 3-link pendulum colliding with the ground.
  • 3D Gyroscope (Gyro): A complex 3D system hitting a wall, testing the limits of 3D contact mechanics.
  • Rope: A high-dimensional system (~400 dimensions) used to test scalability.

Key Insights and Results

  • Interpretable Learning and Parameter Recovery: The model goes beyond simply predicting the next state; it actually "discovers" the physical constants. In the BP5 task, the model recovered true mass ratios (e.g., learning 2.0001 for a true value of 2.0), friction coefficients, and restitution values with remarkable accuracy.
  • Poisson Over Newton: The authors highlight that Newton’s hypothesis (manually reversing velocity) can lead to unrealistic energy gains in 3D systems with rotation. By using Poisson’s hypothesis within an optimization framework, they ensure the learned physics remains energetically sound.
  • Sample Efficiency: Because the model possesses a strong inductive bias—knowing the structure of Lagrangian mechanics and contact constraints—it requires significantly less data than black-box models like standard MLPs or Interaction Networks to reach high accuracy.
  • Energy Conservation: Models using the proposed contact model maintained near-perfect energy conservation during the smooth portions of trajectories, whereas regularized versions or baselines tended to drift or lose energy unrealistically.
  • Differentiable Simulator for Downstream Tasks: Once learned, the model acts as a differentiable physics engine. This is highly valuable for robotics, allowing for gradient-based planning. The authors successfully used the learned model to solve a "Billiards" planning task and a "Disk Throwing" task using simple gradient descent.

Limitations and Future Work

While a significant step forward, the framework has a few limitations:

  • Collision Detection: The system currently assumes a "known collision detection module." It knows when a contact happens but learns what happens during that contact. Future work could involve learning the collision geometry itself.
  • Computational Overhead: Solving a convex optimization problem at every contact point is computationally heavier than using traditional non-differentiable simulators.
  • Scalability Limits: Although it successfully handled a 400-dimensional rope, systems with extremely high mass ratios or "contact-rich" environments (like a bag of sand) remain challenging for this type of acceleration-based simulation.
  • Real-world Data: The current study relies on simulated data. Testing the robustness of this differentiable logic on noisy, real-world sensor data is a logical and necessary next step.

Summary updated

Abstract

The incorporation of appropriate inductive bias plays a critical role in learning dynamics from data. A growing body of work has been exploring ways to enforce energy conservation in the learned dynamics by encoding Lagrangian or Hamiltonian dynamics into the neural network architecture. These existing approaches are based on differential equations, which do not allow discontinuity in the states and thereby limit the class of systems one can learn. However, in reality, most physical systems, such as legged robots and robotic manipulators, involve contacts and collisions, which introduce discontinuities in the states. In this paper, we introduce a differentiable contact model, which can capture contact mechanics: frictionless/frictional, as well as elastic/inelastic. This model can also accommodate inequality constraints, such as limits on the joint angles. The proposed contact model extends the scope of Lagrangian and Hamiltonian neural networks by allowing simultaneous learning of contact and system properties. We demonstrate this framework on a series of challenging 2D and 3D physical systems with different coefficients of restitution and friction. The learned dynamics can be used as a differentiable physics simulator for downstream gradient-based optimization tasks, such as planning and control.
BibTeX
@inproceedings{zhong2021extending,
  author = {Yaofeng Desmond Zhong and Biswadip Dey and Amit Chakraborty},
  title = {{Extending Lagrangian and Hamiltonian Neural Networks with Differentiable Contact Models}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
}
Figure