This paper presents a hierarchical framework for coordinating teams of Autonomous Underwater Vehicles (AUVs) using a combination of Large Language Models (LLMs), mathematical optimization, and robust execution engines. By bridging the gap between high-level human intent and low-level robotic control, the authors aim to make complex multi-robot missions easier to specify and more resilient to the harsh, communication-limited environments of the deep sea.
TL;DR
The authors propose a hierarchical system that uses LLMs to translate natural language mission instructions into optimal plans via Mixed-Integer Linear Programming (MILP), which are then executed by distributed, fault-tolerant Behavior Trees (BTs) on individual AUVs. The framework ensures 100% mission completion in simulation even under sensor failures or vehicle losses, with the Qwen-3-8B model proving particularly adept at structured constraint extraction.
What Problem Does This Paper Address?
Coordinating a fleet of AUVs is notoriously difficult. Operators typically have to write rigid, structured mission files, which is time-consuming and error-prone. Furthermore, the underwater environment is "communication-impoverished"—acoustic links are slow and intermittent.
If a robot fails or a sensor degrades mid-mission, the entire plan often collapses because the system cannot adapt without constant human intervention or high-bandwidth communication. While LLMs offer a promising way to "talk" to robots, they are too slow and unpredictable for real-time safety-critical control. This paper tackles the challenge of providing a user-friendly natural language interface while maintaining the deterministic, reliable execution required for maritime operations.
Methods
The researchers developed a three-layer architecture that separates "topside" reasoning from "seaside" execution:
- Supervision Layer (Topside): An LLM (like Llama-3.1 or Qwen-3) acts as a translator. It takes free-form text (e.g., "Perform task 13 before task 2 with AUVs1") and extracts structured constraints. These constraints are fed into a Mixed-Integer Linear Programming (MILP) optimizer (Gurobi), which generates a mathematically optimal, capability-aware task allocation for the fleet.
- Robotic Agents (RAs): Each vehicle has a dedicated software agent that manages its local state. If a failure occurs (like a lost sonar), the RA determines if it can still perform its remaining tasks. If not, it requests a "selective replan" from the supervisor.
- Execution Layer (Onboard): Tasks are executed using Behavior Trees (BTs). Unlike traditional state machines, BTs are modular and reactive, allowing the robot to handle low-level retries and safety checks deterministically without needing to talk back to the surface.
Datasets
The authors evaluated the system using:
- Constraint Extraction Benchmark: A custom set of 27 test cases (12 simple, 15 complex) designed to test the LLM's ability to handle negations, ordering, and multi-robot assignments.
- ROS-based Simulation: A high-fidelity simulator using Zeno AUV kinematics. They tested three mission scales: Small (2 robots, 8 tasks), Medium (3 robots, 12 tasks), and Large (5 robots, 20 tasks).
Key Insights and Ideas
- LLMs as Translators, Not Controllers: The paper avoids the common pitfall of letting LLMs generate low-level code or control signals. Instead, the LLM only handles semantic interpretation, while a MILP solver acts as a "safety filter" to ensure the resulting plan is physically feasible.
- Behavior Trees for Determinism: By using BTs for onboard execution (see Figure 2), the system maintains the modularity of LLM reasoning while ensuring the robot's actual movements are governed by predictable, pre-defined logic.
- Localized Failure Handling: The "Robotic Agent" concept allows for a "degraded mode." If a robot loses a specific sensor, it doesn't just quit; it checks its task list, completes what it can, and only asks for help for the tasks it can no longer perform. This drastically reduces communication overhead.
Key Results
- Model Performance: In the constraint extraction task, Qwen-3-8B significantly outperformed Llama-3.1-8B, achieving an overall F1 score of 96.2% compared to Llama's 74.5%. Qwen was notably better at generating the valid JSON structures required for the MILP solver.
- Mission Robustness: The framework achieved a 100% Task Completion Rate (TCR) across all tested failure profiles (soft sensor failures, hard vehicle failures, and mixed scenarios).
- Efficiency: Recovery from failures was fast, with mean recovery times ranging from 8.2 to 13.4 seconds, depending on mission size. The communication overhead remained low, typically requiring only three messages to resolve a vehicle failure.
Limitations and Future Work
- Simulation vs. Reality: The current results are based on a ROS simulator. Real-world underwater acoustic noise and localization drift were omitted and will be the focus of future field trials.
- Onboard Compute: The LLM currently runs "topside." The authors suggest that as lightweight models improve, these capabilities could eventually move onto the AUVs themselves.
- Task Complexity: The current task set (waypoints and surveys) is relatively simple. Future work will explore richer task semantics and more complex inter-robot dependencies.
For a detailed breakdown of the LLM performance metrics, see the table below:
Summary updated