MARVEL: Multi-Agent Reinforcement Learning for Constrained Field-of-View Multi-Robot Exploration in Large-Scale Environments

Jimmy Chiun, Shizhe Zhang, Yizhuo Wang, Yuhong Cao, Guillaume Sartoretti

Abstract

Efficient multi-robot exploration is a cornerstone of autonomous search and rescue, mapping, and surveillance. While many existing algorithms assume robots possess 360-degree omnidirectional sensing, real-world platforms—particularly lightweight nano-drones—are often limited by constrained field-of-view (FoV) sensors. This paper introduces MARVEL, a Multi-Agent Reinforcement Learning (MARL) framework designed to address the complexities of directional sensing in large-scale environments. By leveraging graph attention networks and a novel information-driven action pruning strategy, MARVEL enables robots to collaboratively optimize both their spatial positioning and their orientation. Our results demonstrate that MARVEL significantly outperforms state-of-the-art planners in efficiency and reliability, while maintaining zero-shot generalization across diverse team sizes and sensor configurations.

Introduction

Most multi-robot exploration research relies on the assumption that agents are equipped with heavy, high-power sensors like LiDAR, which provide a complete 360-degree view of the surroundings. However, for small-scale robotics, payload and power constraints necessitate the use of lightweight, directional sensors such as monocular or RGB-D cameras.

This constrained FoV introduces a significant challenge: the exploration problem is no longer just about deciding where to go, but also which way to look. This dual requirement expands the action space exponentially. Traditional heuristic methods, such as frontier-based exploration, often struggle in these scenarios because limited visibility makes it difficult to evaluate the utility of frontiers, leading to fragmented maps and redundant coverage. MARVEL addresses these limitations by framing exploration as a sequential decision-making problem solved through reinforcement learning.

The MARVEL Framework

MARVEL utilizes a Centralized Training with Decentralized Execution (CTDE) architecture, allowing agents to learn coordinated behaviors during training while operating independently during deployment.

Graph-Based Representation and Attention

The environment is modeled as a collision-free graph where nodes represent reachable locations. MARVEL employs an encoder-decoder architecture built on stacked masked self-attention layers (Graph Attention Networks). This structure allows agents to process local node features—such as relative position and occupancy—while capturing broader dependencies across the graph. By operating on a graph rather than a fixed grid, the policy becomes inherently scale-invariant.

Information-Driven Action Pruning

To manage the massive action space created by combining movement and heading, MARVEL introduces a deterministic pruning strategy. For every potential neighboring node an agent might move to, the system pre-calculates the number of observable frontiers for various headings. The RL policy is then restricted to choosing from only the top three most informative headings. This pruning makes the reinforcement learning process tractable, allowing the agent to focus on high-level coordination rather than low-level orientation optimization.

Training with Privileged Information

The system is trained using the Soft Actor-Critic (SAC) algorithm. To improve credit assignment and training stability, the critic network is provided with "privileged" ground-truth information, including the full global map and the actions of all other agents. This centralized insight helps the agents learn the value of cooperative strategies that might otherwise be obscured by the partial observability of the environment.

Experimental Results

Performance and Efficiency

MARVEL was evaluated against several established baselines, including Nearest Frontier, MMPF, NBVP, and Learnt-Greedy. In a four-agent scenario on 90m x 90m maps, MARVEL reduced the average trajectory length required for full exploration by 14.2% compared to the next best baseline (NBVP). Furthermore, MARVEL achieved a 100% success rate (defined as exploring 99% of the map within the step limit) across all test scenarios, whereas baselines showed significant fluctuations in reliability.

Generalization and Real-Time Deployment

A key strength of the MARVEL framework is its ability to generalize without retraining. A model trained on a specific sensor range and FoV (e.g., 120°) successfully adapted to different configurations (90° to 180° FoV) and varying team sizes. The decentralized execution is computationally efficient, requiring less than 0.2 seconds per decision. This was validated through physical experiments using a swarm of Crazyflie 2.1 nano drones, demonstrating successful sim-to-real transfer in a controlled flying arena.

Conclusion and Future Work

MARVEL demonstrates that combining graph-based MARL with intelligent action pruning can solve the complex problem of multi-robot exploration with constrained sensors. By explicitly accounting for sensor orientation, the framework prevents the common pitfalls of fragmented mapping and redundant movement.

Currently, MARVEL operates in 2D environments. Future work will focus on extending the framework to full 3D action spaces to better suit the vertical complexities of real-world drone exploration. This will involve integrating altitude-dependent FoV footprints and exploring sparse graph representations to maintain computational efficiency in three-dimensional volumes.

Summary updated

BibTeX
@inproceedings{chiun2025marvel,
  author = {Jimmy Chiun and Shizhe Zhang and Yizhuo Wang and Yuhong Cao and Guillaume Sartoretti},
  title = {{MARVEL: Multi-Agent Reinforcement Learning for Constrained Field-of-View Multi-Robot Exploration in Large-Scale Environments}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2025},
  doi = {10.1109/icra55743.2025.11127700},
}
Figure