Overcoming Discontinuities in Differentiable Robot Simulators

This direction is merged into another direction: discontinuous dynamics modeling. Its last version stays readable and citable here.

Version 1 of 1136 papers160 conceptsfrozen 2026-10-04

In the quest to teach robots complex tasks—like folding laundry, walking on uneven terrain, or performing surgery—researchers have increasingly turned to differentiable simulators. These engines allow us to compute the gradient of a robot's performance with respect to its control parameters or physical design. In theory, this means we can use standard gradient descent to "backpropagate through physics," optimizing a robot's behavior just as we optimize the weights of a neural network.

However, a fundamental mathematical "cliff" stands in the way: contact. In the real world, physics is often non-smooth. One millisecond a foot is in the air (zero force); the next, it hits the ground (massive force). This binary transition creates mathematical discontinuities that break the very gradients we rely on for optimization.

The Problem of the Mathematical Cliff

Standard collision detection algorithms function as non-differentiable black boxes. As noted in Robust Differentiable Collision Detection for General Objects by Baoquan Chen and He Wang, traditional "witness points" jump discretely as geometry changes. These sudden shifts create a "jagged" optimization landscape where gradients are either zero (providing no direction) or infinite (causing the optimizer to explode).

This "crunchy" physics problem is particularly acute in dexterous manipulation. Sizhe Li in DexDeform points out that making and breaking contact creates frequent discontinuities in the computational graph. If a robot hand is even a millimeter away from an object, a standard simulator might report a gradient of zero, leaving the optimizer with no clue that moving closer would be beneficial.

Smoothing the Landscape

The primary strategy to fix this is analytical smoothing—turning the "cliffs" into "slopes." By replacing hard-contact constraints with smooth mathematical surrogates, researchers can provide the optimizer with a continuous signal even before contact occurs.

  1. Barrier Functions and Soft Constraints: Shenao Zhang and Zhaoran Wang introduced Adaptive Barrier Smoothing (ABS) to stabilize policy gradients. By treating contact boundaries as "soft" barriers, they prevent the exponential growth of gradient variance that typically occurs in "stiff" physical simulations.
  2. Differentiable Geometry: Instead of checking for mesh intersections, frameworks like SDRS by Xiaohan Ye and Taku Komura represent robot parts as unions of convex polyhedrons. By applying the Separating Hyperplane Theorem, they achieve provable differentiability even during radical changes in a robot's shape. Similarly, PolyFly by Giuseppe Loianno uses duality theory to transform collision constraints into smooth math, allowing drones to navigate narrow gaps with fluid trajectories.
  3. Long-Range Influence: To solve the "blind spot" where gradients vanish when objects aren't touching, XIAOHAN YE and zherong pan developed an Efficient Differentiable Contact Model with Long-range Influence. This model ensures that gradients are non-vanishing even at a distance, effectively "pulling" the robot toward the contact point.

Hybrid Optimization: When Gradients Aren't Enough

Even with smoothing, the optimization landscape for tasks like folding cloth or tying ropes remains "rugged" and full of local minima. Researchers have found that relying only on gradients is often a recipe for failure.

In Do Differentiable Simulators Give Better Policy Gradients?, H. J. Terry Suh and colleagues argue that first-order gradients can be dangerously biased in chaotic physical systems. They propose the $\alpha$-order gradient estimator, which blends precise first-order gradients with robust zeroth-order (sampling-based) estimates.

This hybrid philosophy is echoed in ThinShellLab by Yian Wang and Zhou Xian. For manipulating thin-shell objects like paper, they couple global sampling (to find the general "neighborhood" of a solution) with local gradient-based refinement (to polish the exact movements). This "search-then-optimize" approach bypasses the discontinuities that would otherwise trap a pure gradient-based solver.

Real-World Transfer

The ultimate test of these smooth models is whether they work on physical hardware. Historically, "softening" physics to make it differentiable meant losing the "hardness" required for real-world accuracy. However, recent work like DiffSim2Real has shown that policies trained purely in differentiable simulations with smooth contact models can successfully deploy on complex robots like the ANYmal D quadruped.

By bridging the gap between the discrete nature of reality and the continuous requirements of calculus, these researchers are turning robot simulation from a mere "checking" tool into a powerful "designing" engine. As summarized in Agile Robotics, the future of the field lies in these hybrid frameworks that combine the precision of differentiable physics with the robustness of stochastic learning.

Papers in this version

Showing 60 of 136 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_overcoming_discontinuities_in_differenti_14d874fc,
  title = {Overcoming Discontinuities in Differentiable Robot Simulators},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/overcoming-discontinuities-in-differentiable-robot-simulators/rdr_14d874fc84144b0d82a0464892656807},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_overcoming_discontinuities_in_differenti_14d874fc_v1,
  title = {Overcoming Discontinuities in Differentiable Robot Simulators},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/overcoming-discontinuities-in-differentiable-robot-simulators/rdr_14d874fc84144b0d82a0464892656807/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}