A Robustness Analysis of Inverse Optimal Control of Bipedal Walking

John R. Rebula, Stefan Schaal, James Finley, Ludovic RighettiarXiv 2021 · arXiv

TL;DR

This paper presents a robust Inverse Optimal Control (IOC) framework designed to infer the underlying cost functions of bipedal walking from observed motion. By testing the algorithm against simulated data under various "real-world" stressors—like sensor noise and model inaccuracies—the authors demonstrate that while the method is remarkably resilient to noise, its primary challenge lies in distinguishing between competing, redundant cost components.

What Problem Does This Paper Address?

In robotics and biomechanics, we often assume that movement (like human walking) is "optimal" with respect to some internal cost function (e.g., minimizing energy or jerk). If we can identify this cost function, we can generate human-like motion for robots or better understand human motor preferences.

However, identifying these costs—a process called Inverse Optimal Control (IOC)—is notoriously difficult. Real-world data is messy: sensors have noise, our mechanical models of the body are simplifications, and we don't always know exactly what task constraints the subject was following. This paper tackles the critical question: How much can we actually trust the cost weights found by IOC when the conditions aren't perfect?

Methods

The authors developed an iterative IOC algorithm and tested it using a simulated 5-link planar walking model (Figure 1).

The 5-link planar walking model used for simulation.

The IOC Pipeline

The core of their approach is an iterative loop (Figure 2):

  1. Demonstration: Start with an "optimal" trajectory generated from known ground-truth weights.
  2. Sampling & Feasibility: Generate random trajectories around the optimal one and project them into the "feasible space" (ensuring they obey physics and task constraints).
  3. Weight Optimization: Find a set of weights that makes the original demonstration cheaper than all the sampled suboptimal trajectories. They use a logarithmic nonlinearity to prioritize trajectories that are very close to the optimal one.
  4. Refinement: Use the new weights to generate a new "suboptimal" trajectory, add it to the set, and repeat until convergence.
The iterative IOC procedure and the various test conditions (noise, bad model, etc.).

Robustness Stress Tests

They evaluated the algorithm under four imperfect conditions:

  • Noisy Trajectory: Adding 5% Gaussian noise to the observed motion.
  • Bad Model: Using a model in the IOC that has 10% more mass in the legs than the one used to generate the data.
  • Bad Speed: Assuming the subject was trying to walk 10% faster than they actually were.
  • Distractor Costs: Adding irrelevant cost components (like torso angle or foot height) to see if the IOC gets "confused."

Datasets

The study relies on simulated datasets generated by the authors' own 5-link model. By using simulation, they have access to the "ground truth" cost weights, which is impossible with real human data. This allows for a direct quantitative assessment of the IOC's accuracy.

Key Insights and Ideas

  • Feasibility as a Filter: One of the most interesting findings is that the IOC algorithm is highly robust to sensor noise. This is because the "feasibility optimization" step naturally smooths out noise by forcing trajectories to obey the laws of physics.
  • The Curse of Non-Uniqueness: The biggest hurdle isn't noise or model error, but "distractor" costs. If multiple combinations of cost components can explain the same motion, the IOC may struggle to find the "true" weights. This highlights a fundamental ambiguity in human movement analysis.
  • Convergence Speed: Under ideal conditions, the algorithm identifies the correct weights almost perfectly within about 25 iterations (Figure 3).
Fig. 3. Results of the IOC algorithm under optimal conditions. The optimal trajectory found is in the left column. To the right are shown the estimated optimal cost weights throughout the process of the IOC . The dashed horizontal lines show the correct cost weights, the solid lines show the candidate weights (wc) throughout the iterative process. Also included is a light gold line representing the error of the trajectory generated using the candidate weights of that iteration. Lastly are shown the estimated weights, which are the candidate weights measured at the iteration with minimal trajectory error throughout the IOC process.

Key Results

  • Ideal Accuracy: Under perfect conditions, the mean weight estimation error was less than 1%.
  • Noise Resilience: Even with 5% noise, the algorithm correctly identified the prioritized cost components, showing qualitative robustness (Figure 4).
  • Error Trade-offs: The "distractor" condition resulted in the highest weight estimation error, confirming that redundant cost components are the primary source of uncertainty (Figure 5).
  • Model/Task Errors: Errors in mass distribution and speed constraints led to higher trajectory errors but still allowed the IOC to identify the dominant cost features.
Summary of trajectory and weight estimation errors across all test conditions.

Limitations and Future Work

  • Model Dependence: The results are highly dependent on the choice of mechanical model. A simple 5-link model cannot capture costs related to specific muscle properties or connective tissues.
  • Redundancy: The authors note that highly correlated costs (like torque squared vs. torque cubed) remain difficult to disambiguate.
  • Future Directions: The team plans to extend the model to include ankles and 3D dynamics (to study lateral stability). They also aim to apply this framework to experimental human data, such as walking on stairs, to see if inferred cost functions can generalize across different tasks.

Summary updated

Abstract

Cost functions have the potential to provide compact and understandable generalizations of motion. The goal of Inverse Optimal Control (IOC) is to analyze an observed behavior which is assumed to be optimal with respect to an unknown cost function, and infer this cost function. Here we develop a method for characterizing cost functions of legged locomotion, with the goal of representing complex humanoid behavior with simple models. To test this methodology we simulate walking gaits of a simple 5 link planar walking model which optimize known cost functions, and assess the ability of our IOC method to recover them. In particular, the IOC method uses an iterative trajectory optimization process to infer cost function weightings consistent with those used to generate a single demonstrated optimal trial. We also explore sensitivity of the IOC to sensor noise in the observed trajectory, imperfect knowledge of the model or task, as well as uncertainty in the components of the cost function used. With appropriate modeling, these methods may help infer cost functions from human data, yielding a compact and generalizable representation of human-like motion for use in humanoid robot controllers, as well as providing a new tool for experimentally exploring human preferences.
BibTeX
@misc{rebula2021robustness,
  author = {John R. Rebula and Stefan Schaal and James Finley and Ludovic Righetti},
  title = {{A Robustness Analysis of Inverse Optimal Control of Bipedal Walking}},
  year = {2021},
  eprint = {2104.12042},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2104.12042},
}
Figure