Precision Instrument Targeting via Image Registration for the Mars 2020 Rover

Gary Doran, David R. Thompson, Tara A. EstlinIJCAI 2016

Introduction

Targeting high-precision instruments on Mars, such as the ChemCam or SuperCam lasers, presents a significant engineering challenge. Currently, scientists on Earth identify targets in rover navigation images and uplink commands. However, mechanical factors like rover "settling," thermal expansion, or hardware "slop" often cause the instrument to miss its millimeter-sized target. Due to the high latency of Mars-Earth communication, a single miss can waste several Sols (Mars days) while waiting for data and sending corrections.

Comparison of the current manual targeting strategy versus the proposed autonomous visual targeting system across multiple Sols.

The proposed "pointing refinement" system addresses this by enabling the rover to autonomously adjust its aim using on-board image registration. By matching real-time images with previous context images, the system corrects pointing errors locally, significantly increasing mission efficiency.

Methods: The Autonomous Registration Pipeline

The system utilizes a landmark-based image registration pipeline optimized for the radiation-hardened, low-power processors typical of space missions (e.g., the 133–150 MHz RAD750).

The image registration pipeline, showing tasks performed on Earth versus those executed autonomously on Mars.
  1. Feature Extraction: The pipeline uses ORB (Oriented FAST and Rotated BRIEF) features. ORB was selected for its speed and low memory footprint compared to traditional features like SIFT or SURF, making it ideal for space-grade CPUs.
  2. Outlier Filtering: To handle "false positives" in landmark matching, the authors introduced a novel filtering step using DBSCAN (density-based clustering). They leveraged the insight that because the camera's field of view is narrow and targets are distant, motion simplifies to nearly pure 2D translation. True matches cluster in "pixel-difference space," while outliers are scattered.
  3. Homography and Safety: After filtering, a homography matrix maps coordinates from the context image to the real-time image. To ensure safety, a Random Forest classifier evaluates the registration quality. If confidence is low, the system falls back to the original pointing to avoid erroneous adjustments.

Key Insights

  • Geometric Simplification: The assumption of 2D translation for narrow-field-of-view instruments allowed for much faster filtering than iterative methods like RANSAC.
  • Hardware Realism: The implementation was designed for the realistic limitations of the RAD750 processor, ensuring the C++ code runs in under one minute on a 150 MHz LEON4 processor.
  • Safety-Critical Design: The use of a post-registration classifier acts as a "vetting" mechanism, a crucial requirement for autonomous space robotics where a wrong decision is worse than no decision.

Results and Performance

The system was evaluated using a massive archive of over 16,500 image pairs from the MSL Curiosity rover.

  • Success Rate: The system achieved over 80% successful targeting when image overlap was at least 33%.
  • Reliability: The Random Forest classifier demonstrated a 97% true-positive rate with only a 0.5% false-positive rate, ensuring high confidence in autonomous adjustments.
  • Robustness: Performance remains stable even with illumination angle changes of up to 60 degrees, covering most same-day observation windows.

Limitations and Future Work

While effective, the system faces challenges with extreme lighting shifts (e.g., morning vs. late afternoon shadows) and featureless terrain like smooth sand. Future developments aim to support "cross-camera" registration, such as matching wide-angle Navigation Camera images to narrow-angle instrument views.

Summary updated

BibTeX
@inproceedings{doran2016precision,
  author = {Gary Doran and David R. Thompson and Tara A. Estlin},
  title = {{Precision Instrument Targeting via Image Registration for the Mars 2020 Rover}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}
Figure