Large Language Model Explainability for Planning Lunar Rover Trajectories under Uncertainty

Michelle Ho, Maximilian Adang, Zachary Booth, David E. Astle, Margaret Hansen, Edward Balaban, Mykel J. KochenderferarXiv 2026 · arXiv

TL;DR

This paper introduces an LLM-based interactive assistant designed to bridge the gap between complex automated trajectory planners and the science teams who use them. By wrapping a Monte Carlo Tree Search (MCTS) planner in a ReAct-based framework with tool access and retrieval-augmented generation, the system achieves 90.9% faithfulness in explaining uncertainty-aware lunar rover paths, a massive 65% improvement over standard zero-shot prompting.

What Problem Does This Paper Address?

Planning a lunar rover mission, like NASA’s VIPER, is a logistical nightmare. Scientists must balance a strict 24-hour time budget, navigation to high-value drill sites, and the physical constraints of a solar-powered vehicle. Crucially, the rover’s speed (Speed Made Good, or SMG) varies based on the sun's position—driving directly toward or away from the low-hanging lunar sun can blind navigation cameras, requiring complex "tacking" maneuvers.

While automated planners like MCTS can quickly find optimal paths through these uncertainties, they are often "black boxes." A science team needs to know why a specific route was chosen and how a 4-hour delay or a new point of interest (POI) might affect mission success. Currently, this verification is done manually, which is too slow for the shifting timelines of active mission planning.

Methods

The authors developed a framework that pairs a rigorous physics-based simulator with a Large Language Model (LLM) assistant.

  1. The Planner: An MCTS-based solver handles a semi-Markov Decision Process (SMDP). It plans "macro-actions" between waypoints (drill sites, POIs) while an A* algorithm handles the fine-grained navigation between them. The simulator accounts for sun-azimuth-dependent speeds and probabilistic delays (wheel slip, comms shadows).
  2. The LLM Assistant: Built using the ReAct (Reasoning and Acting) framework, the assistant doesn't just guess answers. It uses a Tool Catalog to interact with the planner. Tools include Run science station, Change time, Add POI, and Risk assessment (which runs Monte Carlo rollouts to estimate the probability of finishing on time).
  3. Retrieval Layer: To ensure the LLM uses correct domain terminology and mission-specific facts, it uses Retrieval-Augmented Generation (RAG) to search through mission documents and the planner's own source code.
  4. Guardrails: The LLM is prompted to avoid "AI-speak" (like "reward functions" or "value estimates") and instead use the "Operator Vocabulary" familiar to NASA scientists (e.g., "ice quality," "time budget").
Fig. 1: End-to-end flow of our LLM-based science station planning assistant on an example query

Datasets

The system was evaluated using a custom Query Bank of 115 science-team questions. These were categorized into:

  • Plan Comparisons: "How does the plan change if we start 4 hours later?"
  • Feasibility: "What is the risk of not reaching the goal?"
  • Lookup: Questions about mission docs or planner logic.
  • Edge Cases: Ambiguous or impossible requests (e.g., placing a drill site outside the map).

Evaluation was performed using "LLM-as-a-judge" (GPT-4 and Claude Opus), validated against human labels to ensure the judge's scores were trustworthy.

Key Insights and Ideas

  • Explainability via Interaction: Instead of putting the LLM inside the planning loop (where it might make unsafe decisions), the authors use it for post-planning explanation. This allows the LLM to act as a translator for a trusted, deterministic planner.
  • Grounding is Mandatory: Zero-shot LLMs are terrible at this task (26.7% faithfulness). They tend to hallucinate plausible-sounding but physically impossible lunar physics. Only by giving the LLM specific tools to "ask" the planner for data did faithfulness reach acceptable levels.
  • The "Why" remains difficult: While the assistant was excellent at "What" and "How" questions, it still struggled with deep "Why" questions (e.g., "Why did the planner prioritize this site over that one?"). This suggests that even with RAG, some internal logic of MCTS remains hard for LLMs to parse without more explicit symbolic traces.

Key Results

  • 90.9% Faithfulness: The full assistant (ReAct + Tools + RAG) was highly accurate in its factual claims compared to the planner's actual output.
  • Planner Sensitivity: The underlying MCTS planner correctly responded to physical intuition—for example, shifting the start time by 12 hours completely changed the optimal route to account for new sun angles and shadows.
  • Risk Assessment: The Risk assessment tool provided crucial data, showing that while a plan might take 20.15 hours deterministically, uncertainty creates a "tail" of outcomes extending toward the 24-hour limit.
Fig. 3: Planner behavior demonstrated on different a. All panels share the same MDP formulation, number of drill sites selected from 10 candidates, and MCTS hyperparameters. They use the same legend from Figure 2, with blue diamonds as POIs. (a) The original plan takes 20.15 hours without prospecting. (b) A +12 h shift in start time results in a new plan due to different SMGs. (c) 200 Monte Carlo rollouts of (a) under SMG uncertainty, with 99.5% finishing in 24 hours. (d)We added 2 POIs, finishing in 21.7 h. (e) We reran the planner with a new ice map. (f) Enforcing a non-optimal site ordering increased the science station time to 24.45 hours, exceeding the limit. (g) We replanned under new science station bounds.

Limitations and Future Work

  • Hallucinations: Even the best model (Claude Opus) occasionally fabricated details, such as inventing a per-segment breakdown of time slack that the tools never provided.
  • Task Success: While faithfulness was high, "Task Success" (calling the exact right tool for a complex query) was only around 50%. The LLM sometimes chose a "good enough" tool rather than the optimal one.
  • Future Directions: The authors plan to extend the MDP to handle partial observability (where the rover doesn't know the ice concentration until it arrives) and to test the framework with different types of planning algorithms beyond MCTS.

Summary updated

Abstract

Planning planetary rover trajectories under probabilistic uncertainty is a multi-objective optimization problem that science teams currently iteratively solve and verify by hand. This process becomes impractical when timelines and constraints frequently shift throughout the mission planning phase. Sampling-based planners such as Monte Carlo tree search can speed up this process, but their outputs do not preserve traceability to the science objectives that justify each plan, which the science team needs to verify. We propose an interactive assistant based on a large language model (LLM) that can explain planner trajectories using domain vocabulary and re-run the planner under user-specified conditions to compare alternatives. Previous work has used LLMs inside planner decision-making loops, but we explore using them for post-planning explanation, where they are more likely to be trusted since they do not directly influence planner decisions. Our framework pairs a simulator inspired by the NASA Volatiles Investigating Polar Exploration Rover (VIPER) mission with a ReAct queryable interface that calls curated action tools and retrieval functions. We evaluate the assistant on a query bank consisting of plan comparisons, feasibility, and lookup queries. Experiments show that 90.9% of the assistant's responses are faithful to the planner's outputs, a nearly 65% improvement over zero-shot prompting of the same LLM without tool access, demonstrating the potential for trustworthy explainability of uncertainty-aware traverses with an LLM-based system.
BibTeX
@misc{ho2026large,
  author = {Michelle Ho and Maximilian Adang and Zachary Booth and David E. Astle and Margaret Hansen and Edward Balaban and Mykel J. Kochenderfer},
  title = {{Large Language Model Explainability for Planning Lunar Rover Trajectories under Uncertainty}},
  year = {2026},
  eprint = {2611.13292},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.13292},
}
Figure