ContactNets: Learning of Discontinuous Contact Dynamics with Smooth, Implicit Representations

Samuel Pfrommer, Mathew Halm, Michael Posa

Introduction

Robotic manipulation and locomotion are inherently defined by contact. Whether a robot foot strikes the pavement or a gripper secures a package, these interactions involve sudden changes in velocity and complex frictional forces. Traditional deep learning models often struggle with these "sharp" physical events because they are biased toward learning smooth, continuous functions. ContactNets is a novel framework designed to bridge this gap by learning the underlying geometry of contact through implicit representations and a physics-informed loss function.

The Challenge of Discontinuous Dynamics

Standard neural networks are "smoothness-biased," preferring infinitely differentiable functions. However, real-world physics is full of discontinuities: impacts cause instantaneous velocity jumps, and stiction prevents motion until a force threshold is met. When trained to predict these motions directly, standard models often "average out" the jumps, leading to unrealistic artifacts like "ghost forces" or objects sinking into the floor (penetration). While differentiating through physics simulators is a common workaround, these simulators are often numerically unstable and difficult to optimize against.

The ContactNets Framework

ContactNets shifts the learning target from direct state-to-state mapping to learning the implicit representation of contact. Specifically, the model learns:

  1. Signed Distance Functions (SDFs, $\phi$): The distance between an object and a surface.
  2. Contact Jacobians ($J$): How changes in position affect that distance.
The ContactNets training and prediction pipeline. Training leverages an inverse dynamics formulation to find physically consistent contact forces, while prediction uses a forward dynamics simulator.

A Mechanics-Based Loss Function

The core innovation of ContactNets is its training loss, which avoids the need to differentiate through a simulator. Instead, it uses a "hindsight" approach: given an observed motion, it identifies the contact forces ($\lambda$) that best explain that motion according to three pillars of contact mechanics:

  • Normal Complementarity: Forces only exist during contact and only act to push objects apart.
  • Non-penetration: Resulting motions must not allow objects to occupy the same space.
  • Maximum Dissipation: Friction must act to oppose motion and dissipate energy.

By solving a small, efficient convex optimization problem during training, the model finds the most physically plausible explanation for the data, allowing the network to learn accurate geometries without force sensors.

Experimental Evaluation

The researchers evaluated ContactNets on two primary setups:

  1. 1D Toy Example: A point mass falling under gravity, used to demonstrate how standard L2 loss fails to capture velocity jumps compared to the proposed mechanics-based loss.
  2. Real-World 3D Cube Toss: A dataset of 570 tosses of a 10cm acrylic cube onto a wooden surface. The cube's 3D pose was tracked at 148 Hz using AprilTags and high-speed cameras, providing a challenging test case involving corner impacts, sliding, and tumbling.
1D System analysis showing (a) the physical setup, (b) model predictions for position and velocity, and (c) the loss landscape comparing standard L2 loss with the proposed mechanics-based loss.

Key Results

  • Data Efficiency: ContactNets achieved higher positional and rotational accuracy with just 32 tosses than a standard end-to-end baseline could achieve with 256 tosses.
  • Physical Realism: In long-term simulations, the baseline model predicted the cube would sink into the floor by 20% of its width. ContactNets maintained penetration at less than 2%, effectively "discovering" the floor's height and orientation without explicit supervision.
  • Stability: As shown in the 1D toy example, the mechanics-based loss provides a smooth, convex landscape for optimization, whereas standard L2 prediction loss results in a jagged, non-convex landscape that is difficult for gradient descent to navigate.

Conclusion and Future Work

ContactNets demonstrates that embedding physical structure into deep learning models significantly improves their ability to generalize from small datasets while maintaining physical plausibility. While the current model assumes known free-body dynamics and rigid, inelastic bodies, future work aims to extend this framework to soft, deformable objects and to learn dynamics directly from raw video data using keypoint detection.

Summary updated

BibTeX
@inproceedings{pfrommer2020contactnets,
  author = {Samuel Pfrommer and Mathew Halm and Michael Posa},
  title = {{ContactNets: Learning of Discontinuous Contact Dynamics with Smooth, Implicit Representations}},
  booktitle = {Conference on Robot Learning},
  year = {2020},
}
Figure