Inner Product Regularization for Imitation Learning

Benjamin A. Christie, Tianyu Qiu, Amy H. Choi, David Fridovich-Keil, Dylan P. LoseyarXiv 2026 · arXiv

TL;DR

InneR (Inner Product Regularization) is a new technique that makes imitation learning policies significantly more robust by enforcing consistent responses to state perturbations. Instead of simply dampening a robot's sensitivity to all inputs, InneR aligns the "null space" of the policy across local neighborhoods, allowing the robot to ignore irrelevant distractors while remaining highly responsive to task-critical information.

What Problem Does This Paper Address?

Imitation learning—where a robot learns a task from human demonstrations—often suffers from covariate shift. Small execution errors compound, pushing the robot into states it never saw during training. In these unfamiliar states, the robot might latch onto irrelevant features (like the position of a background object) rather than the actual goal, a phenomenon known as causal confusion.

Standard regularization techniques, like penalizing the Frobenius or nuclear norm of the policy Jacobian, try to fix this by making the policy less sensitive to every change in the state. However, this is a blunt instrument: it dampens the robot's response to important task-relevant changes just as much as it dampens noise. The authors argue that a robust policy shouldn't just be "less sensitive"; it should be consistently sensitive to the same directions in state space across local regions.

Methods

The core of InneR is a geometric approach to policy sensitivity.

  1. The Pullback Metric: The authors show that an inner product on the action space (measuring how different two actions are) induces an inner product on the state space through the policy Jacobian. For stochastic policies (like Diffusion Policies), this induced metric is the Fisher Information Matrix (FIM).
  2. Null Space Alignment: The "null space" of this matrix represents directions in the state space that the policy ignores. If a perturbation (like moving a distractor object) falls in this null space, the robot's action doesn't change.
  3. Clustering and Averaging: InneR doesn't enforce a global null space (which would be too restrictive). Instead, it clusters states in a training batch that have similar sensitivity directions.
  4. Eigenvalue Minimization: For each cluster, InneR calculates the average inner product matrix and minimizes its smallest eigenvalues. This forces the states within that cluster to share a common null space—essentially training the robot to ignore the same irrelevant dimensions across a local neighborhood.
Comparison of standard Behavior Cloning (BC) vs. InneR. BC responds inconsistently to perturbations, while InneR aligns sensitivity to ignore irrelevant directions (Δ2) while maintaining response to task-relevant ones (Δ1).

Datasets

The authors evaluated InneR across three distinct environments:

  • Single-Integrator Simulation: A point-mass goal-reaching task used to test robustness against increasing dimensions of irrelevant noise and train-test distribution shifts.
  • Drone Landing Simulation: A Crazyflie quadrotor landing task in a physics simulator (gym-pybullet-drones), where distractor drones were added to the state to test robustness in nonlinear dynamics.
  • Hardware Experiments: A 7-DoF Franka Emika Panda robot performing three tasks: "closing a coffee maker," "stacking two cubes," and "pushing a pot." These tasks included real-world distractors like fruit (apples, oranges, bananas) that were irrelevant to the goal.

Key Insights and Ideas

  • Direction > Magnitude: The "aha moment" is that what a robot ignores is more important than how much it ignores. By regularizing the null space rather than the norm, InneR preserves the policy's expressiveness in important directions.
  • Local Consistency: A robot's focus changes during a task. When picking up a block, the block's pose is critical; once it's in the gripper, that pose becomes constant relative to the hand, and the target location becomes the new focus. InneR’s clustering approach allows the set of "ignored directions" to shift fluidly as the task progresses.
  • Fisher Information as a Bridge: The paper provides a elegant theoretical link showing that Fisher Information is the stochastic equivalent of the Jacobian-based state-space metric used for deterministic policies.

Key Results

  • Real-World Success: On the Franka Panda robot, InneR raised the average success rate of a Diffusion Policy from 18% to 57% in scenarios with multiple distractors, without requiring any additional training data.
  • Robustness to Noise: In drone landing tasks, InneR maintained high success rates even as the state dimension grew with irrelevant distractor data, whereas standard Behavior Cloning (BC) performance dropped significantly.
  • Selectivity: Quantitative analysis showed InneR was 6.5 to 8.4 times more sensitive to task-relevant objects than to distractors, significantly outperforming both BC and standard norm-based regularizers.

Limitations and Future Work

  • Computational Overhead: InneR is computationally expensive, increasing training time by roughly 5x to 6x. This is due to the need for second-order derivatives (differentiating through the Jacobian) and performing eigenvalue decompositions on every batch.
  • High-Dimensional Inputs: The current implementation was tested on state-vector inputs (e.g., object poses). The authors note that extending this to high-dimensional raw inputs like images or point clouds is a major challenge, as computing the full Jacobian for such inputs is currently intractable. Future work may involve using efficient Jacobian-norm estimators to scale InneR to vision-based policies.

Summary updated

Abstract

Robots that learn from demonstrations should extrapolate their policy despite state perturbations such as sensor noise, scene variation, and compounding errors. One approach to this robustness is regularization. Prior regularizers consider each state on its own and limit how sensitive the policy is to any perturbation at that state. By contrast, we explore how the policy's output shifts when given the same perturbation at different states. Our insight is that, locally, the robot should respond to state perturbations in a consistent manner: e.g., if changing an irrelevant object position does not impact the policy at one state, that same change should not shift the policy output at a nearby state. Applying this insight, we develop inner product regularization (InneR). We show that the inner product on action space induces an inner product on state space through the policy Jacobian (the Fisher information for stochastic policies), and that the null space of this inner product governs which state perturbations have a first-order effect on the policy. By clustering states whose inner products are aligned, averaging the inner products within each cluster, and minimizing the smallest eigenvalues of that average, InneR shapes the learned policy so that nearby states share a common null space. InneR enforces consistent responses to perturbations without limiting policy expressiveness, and can be appended to existing imitation learning pipelines for deterministic or stochastic policies. In simulated experiments, InneR makes learned policies more robust to irrelevant state dimensions and shifts between training and test distributions, whereas norm penalties often perform worse than behavior cloning. On a real robot arm performing three manipulation tasks, InneR raises the average success rate of a diffusion policy from 18% to 57% without additional data.
BibTeX
@misc{christie2026inner,
  author = {Benjamin A. Christie and Tianyu Qiu and Amy H. Choi and David Fridovich-Keil and Dylan P. Losey},
  title = {{Inner Product Regularization for Imitation Learning}},
  year = {2026},
  eprint = {2611.17159},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17159},
}
Figure