Bi-Level Image-Guided Ergodic Exploration with Applications to Planetary Rovers

Elena Wittemyer, Ian Abraham

Abstract

We present a method for image-guided exploration for mobile robotic systems. Our approach extends ergodic exploration methods, a recent exploration approach that prioritizes complete coverage of a space, with the use of a learned image classifier that automatically detects objects and updates an information map to guide further exploration and localization of objects. Additionally, to improve outcomes of the information collected by our robot's visual sensor, we present a decomposition of the ergodic optimization problem as bi-level coarse and fine solvers, which act respectively on the robot's body and the robot's visual sensor. Our approach is applied to geological survey and localization of rock formations for Mars rovers, with real images from Mars rovers used to train the image classifier. Results demonstrate 1) improved localization of rock formations compared to naive approaches while 2) minimizing the path length of the exploration through the bi-level exploration.

Introduction

Autonomous exploration is a cornerstone of modern planetary science. With communication delays between Earth and Mars ranging from 5 to 20 minutes, rovers cannot rely on real-time human intervention to identify and investigate geological features of interest. Current exploration strategies often struggle to balance the need for broad area coverage with the requirement for detailed local inspection. This paper introduces Bi-Level Ergodic Trajectory Optimization (BL-ETO), a framework designed to empower mobile robotic systems with the ability to autonomously detect, localize, and explore geological formations by decoupling body movement from sensor orientation.

The Challenge of High-Dimensional Exploration

Traditional exploration often employs Ergodic Trajectory Optimization (ETO). In this paradigm, a robot’s trajectory is optimized such that the time spent in any given region is proportional to the "information density" or "interest" of that region. While effective for simple systems, ETO faces two significant hurdles in planetary contexts:

  1. Computational Complexity: As the degrees of freedom (DOF) increase—such as adding a gimbaled camera to a mobile base—the optimization problem becomes computationally expensive, often exceeding the real-time processing capabilities of flight-ready hardware.
  2. The Visibility Constraint: Sensors have limited fields of view. A rover might drive past a high-value target without "seeing" it if the camera is not actively directed toward the feature.

Methodology: Bi-Level Decomposition

To address these challenges, the researchers propose a bi-level decomposition that separates the exploration task into two distinct but interconnected solvers:

Coarse Solver (The Body)

The coarse solver plans the rover’s path across the 2D terrain. Its primary objective is to cover ground and move the platform toward regions with a high prior probability of containing interesting geological formations.

Fine Solver (The Camera)

Operating simultaneously with the body, the fine solver manages the camera’s orientation (pitch and yaw). It performs a localized ergodic search within its spherical configuration space, "interrogating" the environment to ensure that potential targets are actually captured by the visual sensor.

Image-Guided Feedback Loop

The system integrates a Convolutional Neural Network (CNN) trained on real and simulated Martian imagery sourced from the NASA PDS Imaging Atlas. The CNN classifies terrain features into categories such as "igneous," "sedimentary," or "background." When a target is identified, the rover updates its internal information map in real-time. This update dynamically influences the subsequent trajectories of both the body and the camera, creating a closed-loop system that adapts to new discoveries on the fly.

Experimental Results

The BL-ETO approach was tested in a simulated Martian environment using PyBullet, featuring non-uniform terrain and various rock formations. The method was compared against two baselines: ETO with a fixed camera and ETO with a random camera movement pattern.

The results demonstrated significant improvements in both discovery and efficiency:

  • Detection Rate: BL-ETO successfully localized 58.10% of the rocks in the environment, nearly doubling the performance of the fixed-camera baseline (30.16%) and the random-camera baseline (23.81%).
  • Path Efficiency: Remarkably, BL-ETO achieved these superior detection rates while traveling significantly less distance. The BL-ETO rover covered 755.7 meters, whereas the baseline methods required over 1,576 meters.
  • Real-Time Performance: The CNN achieved an average inference time of 0.139 seconds, proving the feasibility of the bi-level approach for near real-time autonomous decision-making.

Discussion and Future Work

The success of BL-ETO lies in its ability to "freeze" information during specific optimization phases, making the high-dimensional problem tractable. By treating visibility as a searchable parameter rather than a fixed constraint, the rover ensures it does not miss small but critical details.

However, several avenues for future research remain. The current model assumes perfect localization; integrating Visual Simultaneous Localization and Mapping (VSLAM) would be essential for deployment in truly uncharted territory. Additionally, expanding the classifier to handle a wider variety of geological features and environmental hazards would increase the system's robustness. Finally, transitioning from simulation to physical hardware testing in Mars-analog environments is a necessary step toward mission readiness.

Conclusion

Bi-Level Ergodic Trajectory Optimization provides a scalable and efficient solution for autonomous planetary exploration. By intelligently splitting the responsibilities of movement and observation, rovers can maximize scientific return while minimizing energy expenditure and total travel time.

Summary updated

Abstract

We present a method for image-guided exploration for mobile robotic systems. Our approach extends ergodic exploration methods, a recent exploration approach that prioritizes complete coverage of a space, with the use of a learned image classifier that automatically detects objects and updates an information map to guide further exploration and localization of objects. Additionally, to improve outcomes of the information collected by our robot's visual sensor, we present a decomposition of the ergodic optimization problem as bi-level coarse and fine solvers, which act respectively on the robot's body and the robot's visual sensor. Our approach is applied to geological survey and localization of rock formations for Mars rovers, with real images from Mars rovers used to train the image classifier. Results demonstrate 1) improved localization of rock formations compared to naive approaches while 2) minimizing the path length of the exploration through the bi-level exploration.
BibTeX
@inproceedings{wittemyer2023bi,
  author = {Elena Wittemyer and Ian Abraham},
  title = {{Bi-Level Image-Guided Ergodic Exploration with Applications to Planetary Rovers}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=fADiSNpHei},
}
Figure