Shallow Representation is Deep: Learning Uncertainty-aware and Worst-case Random Feature Dynamics

Diego Agudelo-España, Yassine Nemmour, Bernhard Schölkopf, Jia-Jie ZhuarXiv 2021 · arXiv

Abstract

This paper presents a framework for learning nonlinear dynamics using Uncertainty-aware Random Features (URF). By treating a dynamical system as a multi-layer network, the authors establish a mathematical equivalence between finding worst-case system behavior and training deep neural networks via the Frank-Wolfe algorithm. The approach provides expressive, uncertainty-aware models that remain computationally tractable for robust control and stress-testing, bridging the gap between expressive deep learning and reliable uncertainty quantification.

Introduction

In robotics and control, accurate models of system dynamics are essential for safety and performance. While Deep Neural Networks (DNNs) excel at fitting complex, nonlinear data, they often lack reliable uncertainty estimates—knowing what the model doesn't know is critical for avoiding catastrophic failures. Conversely, Gaussian Processes (GPs) provide excellent uncertainty quantification but are often computationally prohibitive for real-time control or overly conservative.

This research introduces the URF dynamics model to address these challenges. The goal is a model that is expressive enough for complex physics, provides a "tube" of plausible trajectories for safety, and remains computationally efficient for identifying worst-case scenarios.

Methodology: The "Shallow is Deep" Insight

The core of the framework is the Uncertainty-aware Random Feature (URF) dynamics model. Instead of a standard DNN, the authors utilize Random Features (RF) to approximate functions in a Reproducing Kernel Hilbert Space (RKHS). This results in a "linear-in-parameter" structure where dynamics are represented as a linear combination of fixed, random nonlinear basis functions.

Key components include:

  • Bayesian Learning: Using Bayesian Linear Regression, the model learns an ellipsoidal uncertainty set for parameters, representing all plausible versions of the dynamics given the available data.
  • Pontryagin’s Minimum Principle (PMP): To find worst-case trajectories, the authors apply PMP. The RF structure ensures the Hamiltonian—a central function in control theory—is linear in the parameters, simplifying optimization.
  • Optimization Duality: The authors prove that while one-step dynamics appear as a "shallow" network, unrolling the system over $N$ steps creates a "deep" network. They demonstrate that an incremental version of PMP is mathematically equivalent to the Frank-Wolfe algorithm acting on the weights of this deep network.

Experimental Evaluation

The approach was evaluated on three simulated nonlinear systems: a synthetic "Source-Spiral" plant, the Van der Pol oscillator, and a damped pendulum. A primary focus was the "small-data" regime (5-15 training rollouts) to test the model's robustness when information is scarce.

Comparison of learned trajectories in small-data vs. large-data regimes. The uncertainty 'tube' (blue shaded area) captures the true trajectory (black) even when data is scarce.

Key Results

  • Uncertainty Quantification: In small-data scenarios, the model produces a wide uncertainty tube. As shown in the experimental results, the true trajectory typically remains within this tube even if the mean prediction deviates. As data increases, this tube naturally shrinks, reflecting higher confidence.
  • Worst-Case Stress Testing: The PMP-based algorithm successfully identifies trajectories that maximize cost functions, allowing for rigorous "stress-testing" of controllers under model uncertainty.
  • Convergence: As the number of training rollouts increases, the gap between the "best-case," "worst-case," and "true" costs vanishes, indicating that the model effectively converges to the true underlying physics.
Convergence of cost estimates as the number of training rollouts increases. The gap between worst-case and best-case costs narrows, converging toward the true cost.

Conclusion and Future Work

The URF framework demonstrates a powerful duality between classical control theory and modern optimization. By lifting states into a high-dimensional feature space, non-convex robust control problems become manageable linear optimizations.

Future research will focus on extending these methods to closed-loop Model Predictive Control (MPC) and automating the selection of kernels for specific physical tasks. While Random Features scale better than Gaussian Processes, handling extremely high-dimensional state spaces remains an area for further exploration through distributionally robust optimization.

Summary updated

Abstract

Random features is a powerful universal function approximator that inherits the theoretical rigor of kernel methods and can scale up to modern learning tasks. This paper views uncertain system models as unknown or uncertain smooth functions in universal reproducing kernel Hilbert spaces. By directly approximating the one-step dynamics function using random features with uncertain parameters, which are equivalent to a shallow Bayesian neural network, we then view the whole dynamical system as a multi-layer neural network. Exploiting the structure of Hamiltonian dynamics, we show that finding worst-case dynamics realizations using Pontryagin's minimum principle is equivalent to performing the Frank-Wolfe algorithm on the deep net. Various numerical experiments on dynamics learning showcase the capacity of our modeling methodology.
BibTeX
@misc{agudeloespana2021shallow,
  author = {Diego Agudelo-España and Yassine Nemmour and Bernhard Schölkopf and Jia-Jie Zhu},
  title = {{Shallow Representation is Deep: Learning Uncertainty-aware and Worst-case Random Feature Dynamics}},
  year = {2021},
  eprint = {2106.13066},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2106.13066},
}
Figure