Bi-Level Image-Guided Ergodic Exploration with Applications to Planetary Rovers

Elena Wittemyer, Ian Abraham

TL;DR

This paper introduces Bi-Level Ergodic Trajectory Optimization (BL-ETO), a method that allows planetary rovers to autonomously explore unknown terrain by balancing "coarse" body movements with "fine" camera scans. By integrating a learned image classifier to update information maps in real-time, the system significantly improves the detection of geological features (like specific rock types) while halving the total distance the rover needs to travel.

What Problem Does This Paper Address?

Autonomous exploration on Mars is hindered by massive communication delays (5–20 minutes), meaning rovers must be able to identify interesting features without waiting for human instructions.

Traditional exploration often uses Ergodic Trajectory Optimization (ETO), where a robot spends time in regions proportional to the "information density" of that area. However, ETO faces two major hurdles:

  1. Computational Complexity: As you add more degrees of freedom (like a rotating camera head on top of a moving body), the math becomes too heavy for real-time processing.
  2. The Visibility Constraint: A rover might drive right past a high-value target if its camera isn't pointed in the right direction at the right time.

The authors aim to create a system that "looks around" intelligently while moving, ensuring it doesn't miss small but important details in a vast landscape.

Methods

The researchers proposed a bi-level decomposition of the exploration problem:

  • Coarse Solver (The Body): Plans the rover's path across the 2D terrain. It focuses on "covering ground" and moving toward regions likely to contain interesting geological formations.
  • Fine Solver (The Camera): While the body moves, the camera head performs its own mini-ergodic search in a spherical space (pitch and yaw). It "interrogates" the local environment to find targets.
  • Image-Guided Feedback Loop: The rover uses a Convolutional Neural Network (CNN) trained on real and simulated Mars imagery to classify rocks as "igneous," "sedimentary," or "background."
  • Dynamic Information Maps: When the CNN identifies a rock, it updates an occupancy grid. This update immediately influences the next set of trajectories for both the body and the camera, creating a closed-loop system that adapts to discoveries on the fly.

Datasets

The authors utilized the NASA PDS Imaging Atlas, a repository of images from actual Mars rovers and satellites.

  • Training Data: They created 21 3D models with textures for sedimentary and igneous rocks. The CNN was trained on 150 images of each rock type, including "edge cases" where rocks were partially out of frame to improve robustness.
  • Simulation Environment: Testing was conducted in PyBullet, using a simulated 100m² Martian environment with non-uniform terrain (hills and plains).

Key Insights and Ideas

  • "Freezing" Information: To make the bi-level optimization tractable, the authors "freeze" the information the camera sees during the fine-tuning phase. This prevents the math from spiraling out of control while the rover is in motion.
  • Visibility as a Search Parameter: Instead of treating the camera as a fixed sensor, they treat the camera's orientation as a searchable space. This acknowledges that where you look is just as important as where you go.
  • Efficiency through Specialization: By splitting the solvers, the rover avoids the "curse of dimensionality." The body doesn't need to worry about the camera's high-frequency jitters, and the camera doesn't need to calculate the physics of driving over a hill.

Key Results

The researchers compared their BL-ETO approach against two baselines: ETO with a fixed camera and ETO with a random camera path.

  • Detection Rate: BL-ETO found 58.10% of the rocks in the environment. In contrast, the fixed camera found only 30.16%, and the random camera found 23.81%.
  • Path Efficiency: Despite finding nearly double the targets, the BL-ETO rover traveled only 755.7 meters, compared to over 1,576 meters for the baseline methods.
  • Speed: The CNN achieved an average inference time of 0.139 seconds, allowing the rover to classify terrain and update its plan in near real-time.

Limitations and Future Work

  • VSLAM Integration: The current model assumes the rover knows its exact position. The authors suggest that integrating Visual Simultaneous Localization and Mapping (VSLAM) would be a necessary step for deployment in truly uncharted territory.
  • Object Diversity: The classifier was limited to two rock types. Future work could involve training on more heterogeneous objects and more complex environmental hazards.
  • Real-World Testing: While the simulation used real Mars imagery, the physical dynamics were simulated. Testing on physical hardware in "Mars-like" Earth environments (like the Mojave Desert) is a logical next step.

Summary updated

Abstract

We present a method for image-guided exploration for mobile robotic systems. Our approach extends ergodic exploration methods, a recent exploration approach that prioritizes complete coverage of a space, with the use of a learned image classifier that automatically detects objects and updates an information map to guide further exploration and localization of objects. Additionally, to improve outcomes of the information collected by our robot's visual sensor, we present a decomposition of the ergodic optimization problem as bi-level coarse and fine solvers, which act respectively on the robot's body and the robot's visual sensor. Our approach is applied to geological survey and localization of rock formations for Mars rovers, with real images from Mars rovers used to train the image classifier. Results demonstrate 1) improved localization of rock formations compared to naive approaches while 2) minimizing the path length of the exploration through the bi-level exploration.
BibTeX
@inproceedings{wittemyer2023bi,
  author = {Elena Wittemyer and Ian Abraham},
  title = {{Bi-Level Image-Guided Ergodic Exploration with Applications to Planetary Rovers}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=fADiSNpHei},
}

Loading related content...

Figure