Coordinated Noise Adversaries in Linear-Quadratic Mean-Field Games

Large populations of interacting agents, such as vehicle swarms, grid-connected devices, or market participants, are often modeled with a mean-field game. This is the limit in which each agent is too small to matter individually and responds only to the population average, called the mean field. Controllers designed this way assume the random disturbances hitting each agent come from a known distribution. Distributionally robust control drops that assumption. It lets an adversary ("nature") pick the disturbance distribution from a ball of distributions within some optimal-transport (Wasserstein) distance of a nominal model, and designs against the worst choice. For a single linear system with quadratic cost, this minimax problem is well understood. The worst-case distribution is Gaussian with an inflated covariance, and linear feedback stays optimal (Taskesen et al., Lanzetti et al.). Nobody has asked what happens to this picture when the noise hits a whole population that is coupled through its average.

We argue the answer hinges on one modeling choice that the obvious extension gets wrong. If each agent's noise distribution is perturbed independently, the perturbations average out in the mean field. The robust mean-field game then collapses almost entirely to the nominal one, which is a population version of the steady-state collapse already documented for robust model predictive control (Distributionally Robust MPC). If instead the adversary may perturb the joint distribution of all agents' noise with the same per-agent budget, it can correlate the "idiosyncratic" noises. This manufactures an order-one shift in the population average that never averages away. Such a coordinated adversary turns the robust problem into a zero-sum linear-quadratic game played on the mean-field state. We propose to prove this split, compute the resulting equilibria, bound the finite-population error, and test the whole account on existing mean-field benchmarks.

Core Claim

Consider $N$ agents with linear dynamics driven by their own control, the population mean, and noise $w^i_t$. A nominal noise law $P_0$ is estimated from $M$ samples. The independent ambiguity model lets each agent's noise law $P^i$ satisfy $W_2(P^i, P_0) \le \epsilon$ separately, where $W_2$ is the 2-Wasserstein distance. The coordinated model constrains the joint law $\mathbf{P}$ of all $N$ noises by

$$W_2(\mathbf{P}, P_0^{\otimes N}) \le \sqrt{N}\,\epsilon,$$

so the total perturbation budget is the same as in the independent model, but the adversary can spend it on correlations.

We conjecture, and will try to prove, three statements. First, under independent ambiguity with zero-mean perturbations and full state feedback, the robust mean-field equilibrium gains coincide with the nominal ones. The worst case only inflates covariance, and the optimal linear gain under full-state feedback does not depend on additive noise covariance. Robustness therefore enters only through the state estimator when observations are partial. Second, under coordinated ambiguity, the worst-case joint law decomposes into two parts. One is a common Gaussian shift of the mean field, which acts like an adversarial control input on the mean-field dynamics. The other is residual covariance inflation of each agent's deviation from the mean. The equilibrium stays linear in each agent's own state and the mean field, and it is obtained from two decoupled Riccati problems: a nominal one for the deviations and a zero-sum one for the mean. Third, deploying this infinite-population coordinated-robust policy in the $N$-agent game gives an approximate saddle point. We expect its error to be roughly $O(1/\sqrt{N})$, the standard mean-field approximation rate, plus a term governed by how fast the estimated radius from $M$ samples covers the true shift.

The genuinely new object is the coordinated-adversary robust mean-field game. The claim that independent ambiguity is nearly vacuous at population scale is also testable and practically important.

Relation to Prior Work

Two single-agent papers supply the method. Taskesen et al. prove Gaussian worst cases and linear optimality, and solve the problem by Frank-Wolfe iterations over LQG subproblems. Lanzetti et al. treat stationary adversaries and show they are less pessimistic than time-varying ones: worst-case cost 1.33 versus 1.50 on a 1D example. We reuse both Gaussian-reduction arguments, but neither paper has population coupling, so the idea of a correlation budget does not arise there.

Three mean-field papers constrain the design. Carmona et al. show zero-sum mean-field-type LQ games have equilibria linear in state and conditional mean, and that gradient descent-ascent finds them. Zaman et al. 2024 solve robust cooperative mean-field control against a deterministic, H-infinity-style disturbance using receding-horizon gradient descent-ascent. Zaman et al. 2021 have an adversary inject control inputs into networked LQ mean-field dynamics.

Our second conjecture says the coordinated distributional adversary reduces to something that looks like these deterministic-adversary games on the mean field. That makes them the right algorithmic templates. It also means a large part of our contribution is deriving the budget and structure of that reduction from a distributional model rather than assuming it.

Guan et al. and Subramanian et al. supply the $O(1/\sqrt{N})$ finite-population arguments, but only for nominal, discrete zero-sum team games. Sanjari and Yüksel show symmetric linear policies are optimal for exchangeable LQG teams under known noise. Their symmetry argument is what we must re-examine, because a coordinated adversary acts on the joint law, and the symmetry has to be preserved on that joint law.

Mahajan et al. motivate the first conjecture. They show optimal linear controllers are invariant to the noise distribution under partially nested information, which is precisely why independent covariance-only perturbations should buy little. The steady-state collapse result in Distributionally Robust MPC is the single-system version of the same phenomenon. We address it directly: it predicts the independent-ambiguity collapse, and it does not apply to the coordinated model, because a mean shift on the population average changes the Riccati problem rather than just the noise level.

Research Program

The theoretical part has four steps. The first proves the independent-ambiguity collapse, with both full-state and partial observations. In the partial-observation case, robustness should appear only as an inflated covariance inside each agent's Kalman-type estimator.

The second step is the core of the project. We would decompose each agent's state into a mean-field component and a deviation component, the population analogue of the state splitting in Mahajan et al.. The goal is to show that the coordinated budget splits optimally between shifting the mean and inflating deviations. This yields a finite-dimensional problem: one scalar split of the budget, plus a zero-sum Riccati equation for the mean.

The third step extends the single-agent Gaussian worst-case argument to the joint law, using exchangeability. The fourth derives the finite-$N$, finite-$M$ deviation bound.

The algorithm is a nested scheme. The outer loop is Frank-Wolfe over the covariance and mean-shift split, in the style of Taskesen et al.. The inner step is a zero-sum LQ solve on the mean field, done either by Riccati iteration or by the projected nested policy gradient of Zhang, Yang, Basar. In that zero-sum LQ setting every stationary point of the policy objective is a global Nash equilibrium.

Where the theory fails, we will report it. The likeliest failure point is the stationary, infinite-horizon coordinated case. Stationarity constraints there may make the adversary's problem non-convex, as Lanzetti et al. already found for a single system.

Baselines

The naive reference point is the nominal LQ mean-field equilibrium. It is computed by the coupled-Riccati method of Carmona et al. using the empirical Gaussian fit to the $M$ samples. It is run on the identical dynamics and sample sets, and it has no tuning budget because it has no robustness parameter. Beating it means lower worst-case cost under coordinated shifts at matched nominal cost.

The first robust competitor is independent per-agent distributional robustness. Each agent runs the Frank-Wolfe controller of Taskesen et al. with the mean field frozen at its nominal trajectory. We give it the same radius grid as our method. This baseline doubles as the ablation that isolates our key ingredient, the correlation budget: the dynamics, radius, and solver are identical, and only the ambiguity set differs.

The strongest existing method is the deterministic-disturbance robust mean-field-type game solved by receding-horizon gradient descent-ascent Zaman et al. 2024. It is the strongest because it already places an adversary on the mean-field dynamics, which our reduction predicts is where the damage occurs. We tune its disturbance-penalty weight over a grid of the same size as our radius grid, with equal gradient-step budgets. No reported numerical result for this method was found on Lacuna; the paper reports only that it stays stable where a multi-agent DDPG variant diverges.

For small $N$ (2 to 8 agents, low-dimensional states), a brute-force semidefinite program on the full joint $N$-agent robust problem serves as ground truth for the saddle value. Beating the H-infinity baseline means a better trade-off curve between nominal cost and worst-case cost under the coordinated adversary, across its whole tuning range rather than at one setting.

Why not just…?

Why not just fit a Gaussian and use the nominal mean-field controller? Single-system evidence says nominal designs degrade under distribution shift. Distributionally Robust MPC reports a 13% closed-loop cost reduction over stochastic MPC when the initial noise law is misestimated. Our own first conjecture, however, says that under independent perturbations the nominal controller is nearly as good as a robust one. The experiments therefore report both regimes, and they concede the nominal controller wherever the independent-perturbation collapse holds.

Why not just run the single-agent robust controller on every agent? This baseline is blind to correlation by construction. It budgets against covariance inflation of its own noise and cannot anticipate a population-wide mean shift. The ablation in the Baselines section measures exactly this gap. If the gap is small, the coordinated model adds nothing, and we would say so.

Why not just treat the mean shift as a deterministic H-infinity disturbance, as in Zaman et al. 2024? This is the real threat, because our reduction predicts the same qualitative structure. The distributional model differs in two ways. It fixes the adversary's budget from data through the Wasserstein radius, rather than through a hand-tuned penalty. It also forces the adversary to trade off between shifting the mean and inflating deviations, whereas an H-infinity budget has no such trade-off. If the tuned H-infinity controller traces the same trade-off curve as ours, the honest conclusion is that our contribution is a principled derivation and a way to set its penalty from $M$ samples, not a better controller. The decisive test below is designed to reveal which of these holds.

Why not just use deep multi-agent RL? The file reports that a multi-agent DDPG variant diverged beyond horizon $T=2$ in exactly this class of games Zaman et al. 2024, so we do not treat it as a serious competitor.

Proposed Experiments

The sanity check at $N=1$ reproduces the 1D stationary example of Lanzetti et al.. Our solver must recover their worst-case costs of 1.33 for the stationary adversary and 1.50 for the non-stationary one.

The main testbeds are the 1D zero-sum mean-field-type game of Carmona et al. and the 2D swarm with 1000 agents from Zaman et al. 2024. On each, we inject nominal noise and three families of shifts: independent covariance inflation, independent non-Gaussian (skewed and uniform) perturbations, and coordinated common shifts. For scale, we use $N$ copies of the 20-state Shell oil fractionator model from Distributionally Robust MPC, coupled through their average. We compare solve time against a MOSEK semidefinite program, noting that paper's report of a more than 50% time reduction for horizons of at least 15 as the reference scale of solver gains.

The metrics are nominal cost, worst-case cost under each shift family, the finite-$N$ saddle gap against the brute-force semidefinite program, and wall-clock time.

Three results are decisive. The first is the gap between the independent and coordinated robust controllers under coordinated shifts, measured as $N$ grows. Our account predicts this gap stays bounded away from zero, while the gap between the independent robust controller and the nominal controller shrinks toward zero. If both gaps vanish, the central claim is false. The second is a plot of finite-$N$ saddle gap against $N$ and $M$ on log axes. The slope in $N$ should be near $-1/2$; the dependence on $M$ is reported as measured rather than assumed. The third is the nominal-versus-worst-case trade-off curve against the fully tuned H-infinity baseline. It determines whether the coordinated distributional model gives better controllers or only a data-driven way to set the H-infinity penalty.