Hypernetwork-based approach for optimal composition design in partially controlled multi-agent systems

Kyeonghyeon Park, David Molina Concha, Hyun-Rok Lee, Chi-Guhn Lee, Taesik LeearXiv 2025 · DOI · arXiv

Abstract

Designing systems where autonomous agents and humans coexist requires optimizing both the number of controllable agents and their underlying reward structures. This paper introduces a hypernetwork-based framework for solving the composition design problem in Partially Controlled Multi-Agent Systems (PCMAS). By leveraging hypernetworks to generate agent policies across a continuous design space, the proposed method avoids the computational burden of repeated training. Evaluated on a large-scale NYC taxi dispatch simulation, the framework achieves near-equilibrium performance and significant system-wide improvements.

Introduction

In many modern infrastructure and service systems, a central designer manages a subset of agents while others operate independently. These Partially Controlled Multi-Agent Systems (PCMAS) are prevalent in autonomous delivery fleets, traffic management, and shared mobility services. A critical challenge for designers is "composition design": determining the optimal number of controllable agents ($N_c$) and their reward parameters ($\alpha_c$) to maximize system utility without destabilizing the environment for independent agents.

The Composition Design Problem

Composition design is inherently a bi-level optimization problem. At the upper level, the designer seeks to maximize a global objective function. At the lower level, agents (both controllable and uncontrollable) engage in a multi-agent reinforcement learning (MARL) process to reach a Nash Equilibrium based on the chosen design parameters. Traditionally, evaluating a single design point requires solving a full MARL problem, which is computationally prohibitive for large-scale systems with high-dimensional design spaces.

Hypernetwork-based Framework

The core innovation of this research is the application of hypernetworks to bypass the need for exhaustive retraining. Instead of training standard policy networks for specific configurations, the authors train a "meta-network"—the hypernetwork.

The proposed hypernetwork-based architecture for PCMAS, featuring actor-critic networks and a mean action network for real-time coordination.

As shown in Figure 2, the hypernetwork takes the system configuration ($N_c, \alpha_c$) as input and generates the weights for the agents' actor and critic networks. This approach exploits the smoothness of the design space, allowing the model to interpolate policies for unseen configurations. To handle large agent populations, the framework utilizes Mean-Field Reinforcement Learning (MFRL). A key addition is the Mean Action Network, which predicts the current average behavior of the crowd in real-time, significantly improving coordination stability compared to standard MFRL approaches that rely on delayed historical data.

Case Study: NYC Taxi Dispatch

The framework was validated using a real-world dataset from the New York City Taxi & Limousine Commission. The researchers modeled a driver repositioning problem during evening peak hours.

The 7x5 grid used for the NYC taxi dispatch simulation, covering Manhattan and parts of Queens.

The simulation environment, depicted in Figure 3, consists of a 7x5 grid covering Manhattan and the route to LaGuardia Airport. The system includes "u-agents" (selfish human drivers) and "c-agents" (controllable drivers managed to improve the overall Order Response Rate). The objective was to find the optimal mix of drivers and reward structures that balances service quality with operational profitability.

Results and Insights

The hypernetwork approach demonstrated several key advantages:

  • Equilibrium Stability: The method achieved NashConv values near zero, indicating that the generated policies were highly accurate approximations of a Nash Equilibrium. This outperformed traditional multi-layer perceptron (MLP) baselines.
  • System Performance: In the NYC simulation, the framework improved the designer’s objective function by up to 13.89% compared to a baseline with no controllable agents.
  • Ablation Analysis: The inclusion of the Mean Action Network was crucial; its removal resulted in a 17.46% performance drop, highlighting the importance of real-time crowd prediction in large-scale coordination.
  • Optimal Control Utility: The study found that controllable agents provide the most significant benefit when the system is under moderate stress. In extremely low or high demand scenarios, the marginal utility of centralized control decreases.

Conclusion

This research provides a scalable and efficient solution for designing complex multi-agent systems. By using hypernetworks to generalize across design configurations, the framework enables rapid optimization of agent compositions. Future work will focus on expanding the flexibility of reward function optimization and scaling the architecture to handle even more heterogeneous agent types.

Summary updated

Abstract

Partially Controlled Multi-Agent Systems (PCMAS) are comprised of controllable agents, managed by a system designer, and uncontrollable agents, operating autonomously. This study addresses an optimal composition design problem in PCMAS, which involves the system designer's problem, determining the optimal number and policies of controllable agents, and the uncontrollable agents' problem, identifying their best-response policies. Solving this bi-level optimization problem is computationally intensive, as it requires repeatedly solving multi-agent reinforcement learning problems under various compositions for both types of agents. To address these challenges, we propose a novel hypernetwork-based framework that jointly optimizes the system's composition and agent policies. Unlike traditional methods that train separate policy networks for each composition, the proposed framework generates policies for both controllable and uncontrollable agents through a unified hypernetwork. This approach enables efficient information sharing across similar configurations, thereby reducing computational overhead. Additional improvements are achieved by incorporating reward parameter optimization and mean action networks. Using real-world New York City taxi data, we demonstrate that our framework outperforms existing methods in approximating equilibrium policies. Our experimental results show significant improvements in key performance metrics, such as order response rate and served demand, highlighting the practical utility of controlling agents and their potential to enhance decision-making in PCMAS.
BibTeX
@misc{park2025hypernetwork,
  author = {Kyeonghyeon Park and David Molina Concha and Hyun-Rok Lee and Chi-Guhn Lee and Taesik Lee},
  title = {{Hypernetwork-based approach for optimal composition design in partially controlled multi-agent systems}},
  year = {2025},
  eprint = {2502.12605},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.12605},
}
Figure