Coherence-based Approximate Derivatives via Web of Affine Spaces Optimization

Daniel Rakita, Chen Liang, Qian WangarXiv 2025 · DOI · arXiv

Introduction

In robotics and physics simulation, calculating derivatives—specifically Jacobians—is a fundamental yet computationally expensive task. Standard approaches like Finite Differencing (FD) and Automatic Differentiation (AD) often struggle with scaling or integration complexity. This paper introduces the Web of Affine Spaces (WASP) Optimization, a method that leverages "coherence"—the similarity between consecutive derivatives in an optimization sequence—to significantly accelerate these computations.

The WASP Method: Exploiting Coherence

The core innovation of WASP is treating derivative estimation as an iterative constrained optimization problem rather than a fresh calculation at every step.

Geometric Intuition

A single Jacobian-Vector Product (JVP) defines an "affine space"—a flat region in high-dimensional space where the true derivative must reside. By maintaining a history of these JVPs, the algorithm builds a "web" of affine spaces.

Geometric intuition of the Web of Affine Spaces (WASP) optimization, showing how the current approximate derivative is constrained by the intersection of previous affine spaces.

When a new input is processed, WASP performs one new JVP to establish a hard constraint. It then solves for the point on this new affine space that is closest to the intersection of previous spaces. This optimization has a closed-form solution via the Karush-Kuhn-Tucker (KKT) system, allowing for extremely fast updates using pre-cacheable matrix operations.

Experimental Evaluation

The researchers tested WASP against synthetic benchmarks and a complex robotics simulation involving a Unitree B1 quadruped with a Z1 manipulator (24 degrees of freedom). The goal was to find a robot pose satisfying specific foot and end-effector placements.

Key Results

  • Performance: In the quadruped task, WASP reached the target pose in 7.0 seconds, vastly outperforming Finite Differencing (50.0s) and PyTorch's Reverse AD (72.3s).
  • Efficiency: The method requires significantly fewer function evaluations than FD, making it ideal for scenarios where function calls are expensive.
  • Implementation: WASP is library-independent and can be implemented in roughly 100 lines of code, requiring only standard forward passes.

Insights and Limitations

The "aha moment" of WASP is the realization that perfect derivatives are often unnecessary for every micro-step of an optimization loop. An approximation that "borrows" information from the previous step is sufficient for convergence and much faster. Using random orthonormal matrices for perturbation directions was found to be critical for maintaining the stability of the "web."

However, the method is an approximation. It is sensitive to step size; if the input changes too drastically between steps (e.g., the robot "teleports"), the coherence is lost, and the web becomes inaccurate. Additionally, while highly effective for robotics-scale problems (hundreds of variables), further research is needed to scale the $n \times n$ matrix operations to the millions of parameters found in deep learning models.

Summary updated

BibTeX
@misc{rakita2025coherence,
  author = {Daniel Rakita and Chen Liang and Qian Wang},
  title = {{Coherence-based Approximate Derivatives via Web of Affine Spaces Optimization}},
  year = {2025},
  eprint = {2504.18790},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.18790},
}
Figure