Team Optimal Control of Coupled Major-Minor Subsystems with Mean-Field Sharing

Jalal Arabneydi, Aditya MahajanarXiv 2020 · DOI · arXiv

Abstract

In this paper, we investigate team optimal control of coupled major-minor subsystems with mean-field sharing. In such a model, there is one major subsystem that directly influences the dynamics of $n$ homogeneous minor subsystems; however, the minor subsystems influence the dynamics of the major subsystem and each other only through their mean behaviour (indirectly). In this model, the major and the minor subsystems are arbitrarily coupled in the cost and have mean-field sharing information structure. We propose a two-step approach. In the first step, we describe a mean-field sharing model with multiple types of minor subsystems as a generalization of the mean-field sharing model of Arabneydi and Mahajan, CDC 2014. In the second step, we use the results obtained in the first step to construct a dynamic programming decomposition to identify optimal strategies for the major and the minor subsystems. We present an example with numerical results to illustrate the approach.

Introduction

Decentralized control involves multiple agents working together to achieve a common objective. However, as the number of agents increases, the complexity of their interactions grows exponentially, often becoming NEXP-complete. To manage this complexity, researchers often employ "Mean-Field" models, which simplify the system by focusing on the average behavior of a large population rather than individual interactions.

While much of the existing literature focuses on Mean-Field Games—where agents act selfishly—this research addresses Team Optimal Control. In this setting, all agents (controllers) cooperate to minimize a global cost function. Specifically, this paper explores the Major-Minor (MF-MM) structure, where a single "major" player (e.g., a service provider) interacts with a large number of "minor" players (e.g., customers).

Problem Formulation: The Major-Minor Structure

In the MF-MM model, the major subsystem directly influences the dynamics of $n$ homogeneous minor subsystems. Conversely, the minor subsystems influence the major player and each other only through their collective mean behavior, known as the mean-field.

The challenge lies in the information structure: "mean-field sharing." Here, all agents observe the state of the major player and the mean-field of the minor players, but they do not necessarily know the exact state of every individual minor player. The goal is to find a decentralized strategy that is team-optimal for the entire system, minimizing a total cost where the major and minor subsystems are arbitrarily coupled.

Methodology: A Two-Step Reduction

The authors propose a novel two-step approach to derive optimal strategies:

  1. Generalization to Multi-Type Models (MF-T): The authors first extend the mean-field sharing model to include multiple "types" of minor subsystems (e.g., different tiers of customers). They demonstrate that by augmenting the state space, this multi-type model can be reduced to a basic mean-field model.
  2. Mapping the Major Player: A key insight of this paper is that the major player can be treated as a unique "type" within this generalized framework. This "change of variables" allows the researchers to apply existing proofs for simpler models to the more complex Major-Minor structure.
  3. Dynamic Programming Decomposition: To solve the decentralized problem, the authors utilize a "virtual coordinator" approach. This coordinator uses common information (the history of the mean-field and the major player's state) to provide "prescriptions" to each agent. By identifying the mean-field as a sufficient statistic, the authors break the problem into sequential optimization steps using Dynamic Programming (DP).

Key Insights

  • Stochastic Mean-Field: In standard mean-field models, the average behavior often becomes deterministic as the population size approaches infinity. However, in the presence of a Major player, the mean-field remains stochastic. The crowd's collective behavior shifts unpredictably in response to the Major player’s random dynamics.
  • Team vs. Game Dynamics: Unlike Nash Equilibrium models where agents are merely reacting to others, the team-optimal approach ensures that agents may take locally sub-optimal actions if it benefits the system's total cost. This cooperation is essential for global efficiency in large-scale infrastructure.

Numerical Results

The theory is validated through a synthetic numerical example involving a service provider (Major) and 100 users (Minors). The provider manages system capacity, while users decide whether to be "active" or "passive."

The results, formalized in Theorem 3, provide the exact DP equations necessary to identify optimal strategies. The model demonstrates how the Major player’s optimal capacity choice and the users' optimal activity levels shift based on the current mean-field. For instance, if the provider has low capacity and user demand is high, the optimal team strategy dictates a jump to high capacity to avoid heavy penalties for unavailable service, even if the transition itself is costly.

Conclusion and Future Work

This paper provides a robust mathematical framework for managing large-scale systems with hierarchical structures. By reducing the Major-Minor problem to a generalized multi-type mean-field model, the authors enable the use of Dynamic Programming to find optimal decentralized strategies.

While the model assumes homogeneity within types and perfect observation of the mean-field, it sets the stage for future research into more complex scenarios, such as systems with noisy observations or continuous state spaces where the computational burden of Dynamic Programming would require further approximation techniques.

Summary updated

Abstract

In this paper, we investigate team optimal control of coupled major-minor subsystems with mean-field sharing. In such a model, there is one major subsystem that directly influences the dynamics of $n$ homogeneous minor subsystems; however, the minor subsystems influence the dynamics of the major subsystem and each other only through their mean behaviour (indirectly). In this model, the major and the minor subsystems are arbitrarily coupled in the cost and have mean-field sharing information structure. We propose a two-step approach. In the first step, we describe a mean-field sharing model with multiple types of minor subsystems as a generalization of the mean-field sharing model of Arabneydi and Mahajan, CDC 2014. In the second step, we use the results obtained in the first step to construct a dynamic programming decomposition to identify optimal strategies for the major and the minor subsystems. We present an example with numerical results to illustrate the approach.
BibTeX
@misc{arabneydi2020team,
  author = {Jalal Arabneydi and Aditya Mahajan},
  title = {{Team Optimal Control of Coupled Major-Minor Subsystems with Mean-Field Sharing}},
  year = {2020},
  eprint = {2012.02401},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2012.02401},
}
Figure