Optimization through Discontinuous Contact Dynamics

692 concepts483 papers6 proposals

The Promise and Peril of Differentiable Physics

Differentiable simulators promise a powerful paradigm shift for robotics and control: by making physics engines mathematically differentiable, we can compute analytical gradients of a trajectory's loss directly with respect to control inputs or physical parameters. In theory, this allows us to use standard gradient descent to optimize complex robotic maneuvers or identify material properties in a fraction of the time required by model-free reinforcement learning.

In practice, however, these optimization landscapes are rarely smooth. When a robot interacts with its environment, it constantly makes and breaks contact. These events introduce sudden, instantaneous changes in forces, velocities, and accelerations. As noted by Zhiao Huang et al. in PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics, discrete collision events create hard mathematical discontinuities that produce non-differentiable jumps in the simulation math. Instead of a smooth, rolling valley where an optimizer can glide toward a minimum, contact-rich environments create optimization landscapes characterized by sharp cliffs, zero-gradient plateaus, and highly non-convex terrain.

Why Local Derivatives Go Blind

To understand why standard gradient descent fails in these environments, we must look at what an analytical first-order gradient actually measures: the infinitesimal slope at a single exact point.

Consider a robotic gripper hovering a fraction of a millimeter above an object. As Filipe de Avila Belbute-Peres points out in End-to-End Differentiable Physics for Learning and Control, standard differentiable physics solvers cannot differentiate through the binary state of whether a contact exists or not. Because the gripper is not yet touching the object, the local derivative treats the space as completely empty. It returns a gradient of zero, remaining entirely "blind" to the massive contact forces that would occur if the gripper moved just a hair further.

This phenomenon leads to what H. J. Terry Suh et al. describe in Do Differentiable Simulators Give Better Policy Gradients? as empirical bias. The optimizer follows a gradient that appears stable locally but points in an incorrect direction because it cannot sense the non-smooth physical boundaries just outside its immediate state.

Furthermore, these landscapes are often highly deceptive. Rika Antonova et al. demonstrate in Rethinking Optimization with Differentiable Simulation from a Global Perspective that rapid contact transitions cause local gradient information to diverge from the global objective, actively pointing away from the true optimum. When combined with naive time-stepping methods in rigid body simulations, these collisions can produce saw-tooth gradients, as shown by Yuanming Hu et al. in DiffTaichi: Differentiable Programming for Physical Simulation, trapping gradient-based solvers in poor local minima.

Smoothing the Landscape via Randomized Sampling

If the fundamental issue is that local derivatives cannot see past the edge of a sharp physical cliff, the solution is to blur the landscape. By replacing an infinitesimal point evaluation with an average over a localized neighborhood, researchers can smooth out mathematical discontinuities into continuous, informative slopes.

This intuition forms the basis of randomized smoothing. H. J. Terry Suh et al. leveraged this in Bundled Gradients through Contact via Randomized Smoothing, demonstrating that averaging gradients over a randomized distribution makes abrupt physical transitions—such as the boundary between sticking and sliding friction—visible to trajectory optimizers.

To balance the precision of analytical derivatives with the robustness of neighborhood exploration, H. J. Terry Suh et al. developed an $\alpha$-order gradient estimator in Do Differentiable Simulators Give Better Policy Gradients?. This hybrid approach blends first-order analytical gradients with zeroth-order sampling. While the first-order gradient tracks the immediate local slope, the zeroth-order samples probe the surrounding neighborhood to detect upcoming contact boundaries, providing a reliable optimization direction even when the local gradient vanishes or misleads.

Hybrid Planning and Surrogate Models

Rather than fixing the gradients of a discontinuous simulator, alternative strategies alter the optimization framework or the physics model itself to circumvent non-smoothness.

One prominent strategy is to decouple the discrete contact decisions from continuous trajectory adjustments. For example, the framework presented by Ramkumar Natarajan et al. in Torque-Limited Manipulation Planning through Contact by Interleaving Graph Search and Trajectory Optimization interleaves discrete graph search with continuous trajectory optimization. The graph search handles the combinatorial challenge of deciding when and where contacts occur, leaving the continuous optimizer to solve smooth trajectories within well-defined contact modes.

Another approach relies on regularizing the underlying dynamics models to make them inherently more gradient-friendly than real-world physics. In PWM: Policy Learning with Multi-Task World Models, Ignat Georgiev demonstrates that intentionally regularizing a learned world model to be smoother than ground-truth physics enables highly efficient gradient-based policy extraction, preventing optimizers from getting trapped by physical cliffs. Similarly, Eliot Xing et al. show in Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulation that incorporating entropy regularization directly into first-order analytic gradients stabilizes policy updates against the chaotic gradient spikes typical of contact-rich environments.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_discontinuous_dynamics_modeling_1069538d,
  title = {discontinuous dynamics modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/discontinuous-dynamics-modeling/rdr_1069538d54d84c9d81e6b9bfa69a3743},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_discontinuous_dynamics_modeling_1069538d_v1,
  title = {discontinuous dynamics modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/discontinuous-dynamics-modeling/rdr_1069538d54d84c9d81e6b9bfa69a3743/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake