Initial Task Allocation for Multi-Human Multi-Robot Teams with Attention-based Deep Reinforcement Learning

Ruiqi Wang, Dezhong Zhao, Byung-Cheol Min

Abstract

Multi-human multi-robot (MH-MR) teams offer significant potential for complex tasks through the collaboration of diverse agents. However, maximizing performance requires sophisticated initial task allocation (ITA) strategies that account for the inherent heterogeneity of team members and tasks. This paper presents a novel formulation of the ITA problem as a Contextual Multi-Attribute Decision-Making Process (CMADP) and proposes AtRL, an attention-based deep reinforcement learning approach. By introducing a cross-attribute attention module, the framework effectively encodes complex dependencies between multiple attributes, leading to superior task allocation in large-scale surveillance scenarios.

Introduction: The Challenge of Heterogeneity

In high-stakes missions such as disaster response or large-scale surveillance, MH-MR teams are characterized by high heterogeneity. Human operators vary in expertise and cognitive resilience, while robots (e.g., UAVs vs. UGVs) possess different speeds, sensor resolutions, and reliability levels. While existing research often focuses on dynamic task reallocation during a mission, the initial setup is critical. A suboptimal initial distribution of labor can prevent a team from ever reaching peak efficiency. The challenge lies in the massive state space and the hidden dependencies between agent capabilities and task requirements.

Methodology: The AtRL Framework

To address these complexities, we propose the AtRL framework, which treats task allocation as a sequential decision process. The architecture is divided into two primary stages:

  1. State Representation Learning: Raw attributes of humans, robots, and tasks are first processed through Recurrent Embedding layers to capture spatial and sequence-dependent features. The core innovation is the Cross-Attribute Attention module. Unlike traditional methods that treat attributes in isolation, this module calculates latent dependencies—for instance, determining how a robot's speed interacts with a human's fatigue resistance—to create a unified context of the team's state.
  2. Policy Learning: The integrated state representation is passed to a Gated Recurrent Unit (GRU). The system is trained using Proximal Policy Optimization (PPO) to maximize a reward function based on overall mission performance.
The AtRL framework architecture, illustrating the flow from multi-attribute inputs to state representation learning and policy learning.

Experimental Evaluation

The framework was evaluated in a massive threat surveillance scenario using the GAMA simulation platform. The environment consisted of a 2km x 2km area with various Points of Interest (POIs) categorized by classification difficulty (Easy, Medium, Hard). The team included UAVs (high speed, low resolution), UGVs (low speed, high resolution), and human operators with varying cognitive abilities and skill levels. The model's generalizability was tested across 500 unseen scenarios.

Results and Discussion

Experimental results demonstrate that AtRL significantly outperforms traditional baselines and standard reinforcement learning approaches.

  • Performance Gains: AtRL and a baseline RL model both achieved significantly higher performance scores than Average (AV) and Random (RA) allocation strategies ($p < 0.001$).
  • The Value of Attention: AtRL consistently outperformed the standard RL model (which lacked the cross-attribute attention module). This confirms that explicitly modeling the relationships between different attribute categories is more effective than simple data concatenation.
  • Intelligent Matchmaking: The attention mechanism effectively acts as a matchmaker, pairing agents based on complementary strengths, such as assigning high-fatigue-risk humans to fast robots to minimize task duration.
Average performance score comparison across different methods, showing AtRL's significant improvement over baseline and standard RL approaches.

Conclusion

The AtRL framework provides a robust solution for the initial task allocation problem in heterogeneous teams. By leveraging cross-attribute attention, the system captures the complex interplay between human factors, robot characteristics, and task attributes. Future work will focus on validating these findings in real-world environments with human-in-the-loop studies and addressing navigation complexities in cluttered terrains.

Summary updated

Abstract

Multi-human multi-robot teams have great potential for complex and large-scale tasks through the collaboration of humans and robots with diverse capabilities and expertise. To efficiently operate such highly heterogeneous teams and maximize team performance timely, sophisticated initial task allocation strategies that consider individual differences across team members and tasks are required. While existing works have shown promising results in reallocating tasks based on agent state and performance, the neglect of the inherent heterogeneity of the team hinders their effectiveness in realistic scenarios. In this paper, we present a novel formulation of the initial task allocation problem in multi-human multi-robot teams as contextual multi-attribute decision-make process and propose an attention-based deep reinforcement learning approach. We introduce a cross-attribute attention module to encode the latent and complex dependencies of multiple attributes in the state representation. We conduct a case study in a massive threat surveillance scenario and demonstrate the strengths of our model.
BibTeX
@inproceedings{wang2023initial,
  author = {Ruiqi Wang and Dezhong Zhao and Byung-Cheol Min},
  title = {{Initial Task Allocation for Multi-Human Multi-Robot Teams with Attention-based Deep Reinforcement Learning}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2023},
  doi = {10.1109/iros55552.2023.10341410},
}
Figure