Stein Variational Ergodic Search

Darrick Lee, Cameron Lerch, Fabio Ramos, Ian Abraham

Abstract

Exploration requires that robots reason about numerous ways to cover a space in response to dynamically changing conditions. However, in continuous domains there are potentially infinitely many options for robots to explore which can prove computationally challenging. How then should a robot efficiently optimize and choose exploration strategies to adopt? In this work, we explore this question through the use of variational inference to efficiently solve for distributions of coverage trajectories. Our approach leverages ergodic search methods to optimize coverage trajectories in continuous time and space. In order to reason about distributions of trajectories, we formulate ergodic search as a probabilistic inference problem. We propose to leverage Stein variational methods to approximate a posterior distribution over ergodic trajectories through parallel computation. As a result, it becomes possible to efficiently optimize distributions of feasible coverage trajectories for which robots can adapt exploration. We demonstrate that the proposed Stein variational ergodic search approach facilitates efficient identification of multiple coverage strategies and show online adaptation in a model-predictive control formulation. Simulated and physical experiments demonstrate adaptability and diversity in exploration strategies online.

Introduction

Robotic exploration is a complex balancing act. A robot must decide where to go to gather the most information while navigating obstacles and adapting to changing conditions. Traditional "myopic" methods focus only on immediate information gain, which often leads to robots getting stuck in local minima.

While Ergodic Search methods offer a more robust alternative by optimizing trajectories so that the time spent in an area matches its "information density," they typically only produce a single optimal path. In dynamic environments, a single plan is brittle; if an obstacle moves or a sensor fails, the robot must recompute from scratch. Furthermore, the underlying mathematics of ergodic search is non-convex, meaning many high-quality, "locally optimal" paths are ignored by standard solvers. The challenge lies in efficiently finding a diverse set of these paths and enabling the robot to switch between them in real-time.

Methodology: Control as Inference

To address these challenges, we introduce Stein Variational Ergodic Search (SVES). This framework treats robotic exploration as a probabilistic inference problem, defining a likelihood function where "good" trajectories are those with low ergodic cost (high coverage). To sample from this distribution, we utilize Stein Variational Gradient Descent (SVGD).

Key components of the SVES framework include:

  • Ergodic Metric: A spectral (Fourier-based) cost function measuring how well a trajectory covers a target distribution.
  • SVGD Particles: Instead of a single path, the algorithm optimizes a "cloud" of trajectories (particles) simultaneously.
  • Repulsive Force: A core feature of SVGD is a kernel-based repulsive force that prevents "mode collapse," ensuring trajectories spread out to find different exploratory strategies.
  • Markov RBF Kernels: To handle the high dimensionality of trajectories, we use a structured kernel that leverages the Markov property of state sequences, preventing vanishing gradient issues.
Comparison of trajectory distributions using standard RBF kernels, Markov RBF kernels, and vanilla optimization.

By combining these elements, SVES turns the non-convexity of the ergodic metric into a feature, allowing the robot to discover a "multi-modal" landscape of valid exploration strategies. This process is highly parallelizable; on a GPU, the gradient updates remain constant-time regardless of the number of trajectories being optimized.

Experimental Evaluation

We evaluated SVES across several simulated and physical environments:

  • Simulated 2D Forest: A 100m x 100m domain with tree-like obstacles.
  • 3D Aircraft Dynamics: A complex non-linear model navigating a bimodal information distribution.
  • Sonar-based Mapping: An underwater mapping task using bathymetric data where the robot chooses sampling locations based on a Gaussian Process.
  • Physical Drone Experiment: Validation using a Crazyflie 2.1 nano-quadcopter in a 3D environment with physical obstacles.

Key Results and Discussion

SVES successfully maintains a diverse set of trajectories where vanilla ergodic search collapses to a single solution. This diversity was quantified using the determinant of the kernel matrix, which remained stable for SVES throughout optimization. In mapping tasks, SVES outperformed myopic planners across all metrics (SMSE, RMSE, MNLL), proving that long-term coverage reasoning leads to superior Gaussian Process models.

Physical drone experiment showing the diversity of coverage trajectories generated by SVES.

Real-time performance is a significant advantage: on an NVIDIA RTX 3080, a gradient step took only ~67 microseconds. This speed allows for online adaptation; in dynamic simulations, the robot could switch between pre-computed diverse plans instantly when its primary path was blocked.

While the computational cost of the ergodic metric grows with spatial dimensionality, and performance can be sensitive to hyperparameter selection, SVES represents a significant step toward robust, multi-strategy robotic exploration. Future work will explore order reduction techniques to handle even higher-dimensional tasks more efficiently.

Summary updated

Abstract

Exploration requires that robots reason about numerous ways to cover a space in response to dynamically changing conditions. However, in continuous domains there are potentially infinitely many options for robots to explore which can prove computationally challenging. How then should a robot efficiently optimize and choose exploration strategies to adopt? In this work, we explore this question through the use of variational inference to efficiently solve for distributions of coverage trajectories. Our approach leverages ergodic search methods to optimize coverage trajectories in continuous time and space. In order to reason about distributions of trajectories, we formulate ergodic search as a probabilistic inference problem. We propose to leverage Stein variational methods to approximate a posterior distribution over ergodic trajectories through parallel computation. As a result, it becomes possible to efficiently optimize distributions of feasible coverage trajectories for which robots can adapt exploration. We demonstrate that the proposed Stein variational ergodic search approach facilitates efficient identification of multiple coverage strategies and show online adaptation in a model-predictive control formulation. Simulated and physical experiments demonstrate adaptability and diversity in exploration strategies online.
BibTeX
@inproceedings{lee2024stein,
  author = {Darrick Lee and Cameron Lerch and Fabio Ramos and Ian Abraham},
  title = {{Stein Variational Ergodic Search}},
  booktitle = {Robotics: Science and Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=9UtAbCZ4AL},
}
Figure