Sequential Bayesian Optimization for Adaptive Informative Path Planning with Multimodal Sensing

Joshua Ott, Edward Balaban, Mykel J. Kochenderfer

TL;DR

This paper introduces a new framework for robots exploring unknown environments using multiple sensors with varying costs and accuracies. By formulating the problem as a belief Markov Decision Process (MDP) and solving it with sequential Bayesian optimization, the authors' approach doubles the average reward and halves the environment mapping error compared to previous state-of-the-art methods.

What Problem Does This Paper Address?

Imagine a Mars rover equipped with a low-energy camera and a high-energy drill. The camera gives a fuzzy idea of what's underground, while the drill provides a perfect measurement but consumes massive amounts of power. How should the rover decide where to move and which sensor to use to maximize scientific discovery without running out of batteries?

This is the Adaptive Informative Path Planning with Multimodal Sensing (AIPPMS) problem. While standard path planning focuses on just moving, AIPPMS requires an agent to reason jointly about movement and sensing. It is a notoriously difficult (NP-hard) problem because the agent must balance "exploration" (gathering info to reduce uncertainty) with "exploitation" (visiting known high-value targets) under strict resource constraints.

Methods

The authors propose a sophisticated three-part solution:

  1. Belief MDP Formulation: Instead of just tracking the robot's physical location, the "state" includes the robot’s internal "belief" about the entire environment.
  2. Gaussian Process (GP) Beliefs: The environment is modeled using Gaussian Processes. This allows the robot to exploit spatial correlations—learning about one spot tells the robot something about the neighboring spots, even if it hasn't visited them yet.
  3. MCTS-DPW Solver: To handle the complex, continuous nature of these beliefs, the authors use Monte Carlo Tree Search with Double Progressive Widening (MCTS-DPW). Unlike "greedy" algorithms that only look one step ahead, this approach allows the robot to simulate many potential future paths and sensing actions, choosing the sequence that maximizes long-term "Mutual Information" (a mathematical measure of how much uncertainty is reduced).

Datasets

The researchers evaluated their approach on two main benchmarks:

  • Information Search RockSample (ISRS): A variation of a classic robotics benchmark where an agent moves through a grid to find "good" rocks while using beacons to sense their quality from a distance.
  • Rover Exploration (New): A new benchmark introduced in this paper. It simulates a planetary rover mission where the agent must choose between a noisy spectrometer and an expensive, accurate drill to identify unique measurement types across a spatially correlated map.

Key Insights and Ideas

  • Lookahead Matters: Greedy algorithms often fail in multimodal sensing because they don't want to "pay" the high cost of an accurate sensor right now. By looking several steps into the future, the MCTS-DPW solver realizes that an expensive sensor "investment" now leads to much higher rewards later.
  • Spatial Reasoning: By using Gaussian Processes, the agent becomes much more efficient. It doesn't need to drill every single hole; it can use a few high-quality samples to infer the state of the surrounding area.
  • Information as Reward: The reward function isn't just about finding "gold"; it explicitly rewards the agent for reducing the total variance (uncertainty) of its map. This encourages the robot to take sensing actions to become certain about a target's value before committing the energy to visit it.

Key Results

The proposed method significantly outperformed existing baselines (like POMCP and POMCP-GCB):

  • Reward: In the ISRS experiments, the MCTS-DPW approach more than doubled the average reward received in almost every scenario.
  • Accuracy: It reduced the Root-Mean-Square Error (RMSE) of the environment belief by 50%.
  • Robustness: In the Rover Exploration task, the agent was far better at avoiding "redundant" samples (drilling where it had already been) compared to standard raster-scan policies, especially as sensor noise increased.

Limitations and Future Work

The primary trade-off for this performance is computational speed. Because Gaussian Processes require complex matrix inversions ($O(n^3)$ complexity) and MCTS explores many future branches, the planning time is significantly higher than greedy methods (roughly 0.5 seconds per plan vs. 0.01 seconds for simpler methods).

The authors suggest several paths forward:

  • Scaling: Integrating more efficient GP representations (like sparse GPs) to handle much larger maps.
  • Adaptive Resolution: Allowing the robot to change the "resolution" of its map on the fly to save computation.
  • Parameter Learning: Enabling the robot to reason about and learn the underlying parameters of the Gaussian Process (like the "length-scale" of spatial correlation) while it explores.

Summary updated

BibTeX
@inproceedings{ott2023sequential,
  author = {Joshua Ott and Edward Balaban and Mykel J. Kochenderfer},
  title = {{Sequential Bayesian Optimization for Adaptive Informative Path Planning with Multimodal Sensing}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2023},
  doi = {10.1109/icra48891.2023.10160859},
}

Loading related content...

Figure