Evaluating Depth-Augmented Visual Features for Cross-Dataset Lunar Traversability Estimation

Jan Baturo, Bartosz PtakarXiv 2026 · arXiv

TL;DR

This paper investigates whether augmenting visual features with geometric data (like depth and surface normals) improves a lunar rover's ability to estimate terrain traversability across different datasets. Surprisingly, the researchers found that a strong visual-only baseline is remarkably robust, and the choice of training data source is a far more critical factor for success on the real lunar surface than adding extra geometric sensors.

What Problem Does This Paper Address?

Autonomous mobility is a prerequisite for long-term lunar exploration. Because of significant communication delays and frequent line-of-sight occlusions, rovers must be able to judge for themselves whether the terrain ahead—filled with craters, loose regolith, and jagged rocks—is safe to cross.

The core challenge is the domain gap. Most models are trained on terrestrial "lunar analogues" (Earth-based facilities that mimic the Moon). However, the harsh, unfiltered lighting and unique textures of the real lunar surface are significantly different from Earth-based simulants. Researchers hypothesized that while "photometric" cues (lighting and color) change between domains, "geometric" cues (the physical shape of a rock or the steepness of a slope) should remain invariant, potentially bridging the gap between Earth-based training and Moon-based deployment.

The same real lunar scene shown as visual, surface normals, and predicted traversability.

Methods

The authors built upon a weakly supervised framework called W-RIZZ (and its adaptation CHUNGUS). Instead of requiring thousands of absolute "safety scores," this method uses sparse pairwise annotations: a human simply points to two spots in an image and says "Point A is safer than Point B."

Key technical components include:

  • Visual Backbone: They used DINOv2, a self-supervised foundation model, to extract dense visual descriptors.
  • Upsampling: They employed AnyUp to increase the resolution of these features while maintaining semantic consistency.
  • Geometric Fusion: They tested a "naive" fusion strategy, concatenating the 384-dimensional visual vector with various depth-derived channels:
  • Depth: Raw distance values.
  • Normals: 3D vectors encoding surface orientation.
  • Geometric: A suite of features including slope magnitude, local roughness, and Z-score anomalies (to find outliers like rocks).
  • Edge Deployment: The models were optimized using TensorRT and tested on an NVIDIA Jetson AGX Orin to ensure they could run in real-time on actual rover hardware.

Datasets

The study utilized three distinct datasets to test cross-domain generalization:

  1. LunaPolaris: An indoor analogue facility using a stereo camera.
  2. In-House (OTRK Kąkolewo): A facility featuring diverse rock formations, captured using an indirect Time-of-Flight (ToF) camera.
  3. CE-3/CE-4: Real lunar surface imagery from the Chinese Chang’E-3 and Chang’E-4 missions. Since these lacked aligned stereo data, the authors used neural matching (S2M2) to estimate depth maps.

Key Insights and Ideas

  • The Geometry Myth: Contrary to expectations, adding geometry did not consistently outperform the visual-only baseline. In many cases, the improvements were smaller than the "noise" or variance between different training runs (seeds).
  • Source Dominance: The relevance of the training data was more important than the modality. For example, models trained on the LunaPolaris analogue transferred much better to real lunar imagery than those trained on the In-House facility, regardless of whether they used depth data.
  • The Power of Foundation Models: The robustness of the visual-only baseline suggests that foundation models like DINOv2 already capture significant structural information from images alone, making simple geometric augmentation redundant in many scenarios.
  • Rigorous Evaluation Matters: The authors emphasize the importance of multi-seed evaluation. They noted that a single "lucky" training run might show a 30% improvement, but that advantage often disappears when averaged over ten runs.

Key Results

The primary metric used was the Human Disagreement Rate (HDR), where a lower score indicates the model's rankings better match human judgment.

  • Quantitative: In cross-domain transfer from the In-House facility to LunaPolaris, adding raw depth reduced the HDR from 0.119 to 0.107. However, when transferring to the real Moon (CE-3/4), the visual-only model (HDR 0.102) often performed as well as or better than the geometric versions.
  • Qualitative: While visual models sometimes struggled with subtle terrain variations, depth-enhanced models produced more "cohesive" traversability regions in real lunar scenes.
  • Speed: On the Jetson AGX Orin, the visual and depth variants achieved nearly 29 FPS (TensorRT FP16), making them highly suitable for real-time navigation. Adding surface normals or complex geometric descriptors dropped the throughput to roughly 19-20 FPS due to the extra preprocessing required.
Change in HDR relative to the visual-only baseline. Values below zero indicate improvement.

Limitations and Future Work

The authors acknowledge that their "naive" fusion (simple concatenation) might be too simple. Future work could explore more sophisticated architectures, such as cross-attention mechanisms, to better integrate visual and geometric signals.

Additionally, the study relied on a single human annotator, which introduces subjective bias. The researchers plan to move toward "robot-centric" validation, where the model is integrated with a path-planning algorithm and tested on a physical rover to measure actual performance metrics like power consumption and obstacle avoidance success.

Summary updated

Abstract

Autonomous mobility is essential for sustained lunar surface operations, as communication delays and line-of-sight occlusions limit continuous teleoperation. Although recent traversability estimation methods increasingly rely on deep learning, the domain gap between terrestrial analogue environments and real lunar terrain can limit their generalisation, and real lunar traversability labels are scarce, making fully supervised training difficult. To address these challenges, we build a weakly supervised framework trained on sparse pairwise annotations and study whether augmenting visual features with depth-derived geometric information: depth, surface normals, and combined geometric descriptors. We evaluate the models in both in-domain and cross-domain settings using two terrestrial analogue datasets and the real lunar Chang'E PCAM dataset. Contrary to expectations, we find that naive geometric augmentation yields no consistent improvement over a strong visual-only baseline in either setting, with effects small relative to seed-to-seed variance. Instead, the choice of training source emerges as the dominant factor in transfer quality: models trained on domain-relevant sources transfer substantially better to real lunar imagery than the geometric fusion strategy improves any single model. Our evaluation runs onboard a resource-constrained edge device and provides a controlled baseline for designing more expressive multimodal fusion architectures in future lunar perception systems.
BibTeX
@misc{baturo2026evaluating,
  author = {Jan Baturo and Bartosz Ptak},
  title = {{Evaluating Depth-Augmented Visual Features for Cross-Dataset Lunar Traversability Estimation}},
  year = {2026},
  eprint = {2611.03430},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.03430},
}
Figure