Initial Task Allocation for Multi-Human Multi-Robot Teams with Attention-Based Deep Reinforcement Learning

Ruiqi Wang, Dezhong Zhao, Byung-Cheol Min

TL;DR

Researchers have developed AtRL, an attention-based Deep Reinforcement Learning framework designed to solve the "Initial Task Allocation" (ITA) problem for teams composed of multiple humans and multiple robots. By using a novel cross-attribute attention mechanism, the system learns to intelligently pair specific humans, robots, and tasks based on their unique strengths and weaknesses, significantly outperforming traditional random or averaged assignment strategies.

What Problem Does This Paper Address?

In complex missions like disaster response or large-scale surveillance, we often use Multi-Human Multi-Robot (MH-MR) teams. These teams are "highly heterogeneous"—meaning every member is different. One human might be an expert but tires easily; one robot might be fast but has a low-resolution camera.

Most existing research focuses on re-allocating tasks once a mission is already underway. However, the authors argue that the initial setup is the most critical phase. If you start with a poor distribution of labor, the team may never reach peak efficiency, even with mid-mission adjustments. The challenge is that the "state space" for these assignments is massive and full of hidden dependencies (e.g., how a specific robot's camera quality interacts with a specific human's skill level for a specific type of difficult task).

Methods

The authors formulate the problem as a Contextual Multi-Attribute Decision-Making Process (CMADP). To solve it, they propose the AtRL framework, which consists of two main stages:

  1. State Representation Learning:
    • Recurrent Embedding: Raw data about humans, robots, and tasks are fed into Recurrent Neural Networks (RNNs) to capture spatial and sequence-dependent features.
  2. Cross-Attribute Attention: This is the core innovation. Instead of just looking at attributes in isolation, this module uses an attention mechanism to find "latent dependencies." It calculates how much one attribute (like a robot's speed) should "attend" to another (like a human's fatigue resistance) to create a unified, high-level understanding of the team's context.
  3. Policy Learning:
    • The learned representation is passed to a Gated Recurrent Unit (GRU) and trained using Proximal Policy Optimization (PPO). The goal is to maximize a reward function based on the team's overall performance in identifying threats.

Datasets

The researchers evaluated their model using a custom-built simulation environment on the GAMA platform.

  • Scenario: A 2km x 2km surveillance area containing various Points of Interest (POIs).
  • Agents: The team included UAVs (fast, low-res images), UGVs (slow, high-res images), and human operators with varying levels of "cognitive ability" (fatigue resistance) and "operational skill."
  • Tasks: POIs were categorized by location and classification difficulty (Easy, Medium, Hard).
  • Evaluation: The model was tested on 500 "unseen" scenarios to ensure it learned general strategies rather than just memorizing specific setups.

Key Insights and Ideas

  • The "Initial" Advantage: Correct initial allocation leverages the intrinsic heterogeneity of the team from second one, leading to maximum performance in a shorter period.
  • Attention as a Matchmaker: The cross-attribute attention module acts as a sophisticated matchmaker. It doesn't just see "a human" and "a robot"; it understands that a human with low cognitive resilience should be paired with a fast robot (UAV) to minimize the time they spend on a single task, thereby reducing fatigue.
  • Complexity Scaling: The performance gap between AtRL and standard methods grows as the team gets larger and the tasks get harder. This suggests that as systems become more complex, "smart" allocation becomes a necessity rather than a luxury.

Key Results

  • Statistical Superiority: AtRL and a basic RL ablation model both significantly outperformed Average (AV) and Random (RA) baselines ($p < 0.001$).
  • The Power of Attention: AtRL significantly outperformed the standard RL model (which lacked the cross-attribute attention module). This proves that explicitly modeling the relationships between different categories of attributes is more effective than just concatenating data.
  • Efficiency: Learning curves showed that AtRL achieved higher rewards with a more stable and efficient learning pattern compared to standard RL.

Limitations and Future Work

  • Simulation vs. Reality: The study was conducted in a controlled simulation. Real-world human behavior is often more unpredictable than the mathematical models used here (e.g., the Sigmoid pattern for task difficulty).
  • Environmental Constraints: The current scenario is an open space. The authors acknowledge that "cluttered environments" (like urban ruins or forests) would add navigation complexities not yet accounted for.
  • Human-in-the-Loop Validation: Future work aims to conduct user studies with real humans to validate if the AI's "optimal" assignments actually feel optimal to the people doing the work.

Summary updated

Abstract

Multi-human multi-robot teams have great potential for complex and large-scale tasks through the collaboration of humans and robots with diverse capabilities and expertise. To efficiently operate such highly heterogeneous teams and maximize team performance timely, sophisticated initial task allocation strategies that consider individual differences across team members and tasks are required. While existing works have shown promising results in reallocating tasks based on agent state and performance, the neglect of the inherent heterogeneity of the team hinders their effectiveness in realistic scenarios. In this paper, we present a novel formulation of the initial task allocation problem in multi-human multi-robot teams as contextual multi-attribute decision-make process and propose an attention-based deep reinforcement learning approach. We introduce a cross-attribute attention module to encode the latent and complex dependencies of multiple attributes in the state representation. We conduct a case study in a massive threat surveillance scenario and demonstrate the strengths of our model.
BibTeX
@inproceedings{wang2023initial,
  author = {Ruiqi Wang and Dezhong Zhao and Byung-Cheol Min},
  title = {{Initial Task Allocation for Multi-Human Multi-Robot Teams with Attention-Based Deep Reinforcement Learning}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2023},
  doi = {10.1109/iros55552.2023.10341410},
}

Loading related content...

Figure