When two cars meet at a highway on-ramp, one of them gives way. Game-theoretic trajectory predictors model this as each driver optimizing its own cost while sharing a "don't collide" requirement. A useful special case is the potential game: a game in which every agent's incentive to change its own plan equals the change in one shared scalar function, so finding an equilibrium reduces to minimizing that function. Existing learned versions of these models treat collision avoidance as a soft penalty, a repulsive cost term that a network tunes alongside everything else. That choice makes training easy. It also blurs the quantity we most want to learn, namely how the burden of a shared constraint is split between agents. Penalty stiffness and agent priority become entangled, and predicted trajectories can still overlap.
We propose to keep collision and road constraints hard and to model asymmetric priority explicitly through per-agent weights. The game is a weighted constrained potential dynamic game, which recent work shows collapses to a single constrained optimal-control problem. We embed it as a differentiable layer. Gradients are obtained by implicit differentiation: the Implicit Function Theorem is applied to the solution's optimality conditions, the Karush–Kuhn–Tucker (KKT) conditions, instead of backpropagating through solver iterations. The central point is that in a weighted potential game, the KKT multiplier on a shared constraint, combined with the learned weights, says how much each agent pays to satisfy that constraint. Making these quantities exact and differentiable turns "who yields, and by how much" into a learnable, identifiable, and inspectable output instead of a by-product of penalty tuning.
Core Claim
A network that predicts per-agent weights and per-pair safety margins, feeding a hard-constrained weighted potential game layer differentiated through its active KKT system, will do three things. It will match soft-penalty game layers on displacement error. It will produce zero predicted overlaps by construction. And it will recover the underlying priority weights far more accurately, especially from interaction-heavy data where constraints are active.
The layer solves
$$ y^*(\theta) = \arg\min_{y} \; \Phi(y;\theta) = \sum_{i=1}^{N} w_i(\theta)\, J_i(y_i;\theta) + C(y;\theta) \quad \text{s.t.} \quad g(y;\theta) \le 0,\;\; x_{t+1} = f(x_t,u_t), $$
where $y$ stacks all agents' states $x_t$ and controls $u_t$ over the horizon, and $w_i > 0$ is agent $i$'s learned weight on its private cost $J_i$ (target speed, comfort, lane keeping). $C$ is a symmetric shared interaction cost, and $g$ collects hard shared constraints such as pairwise separation with learned margin $d_{ij}(\theta)$, road bounds, and actuator limits. Here $\theta$ is the network output.
The structural lemmas of Strategic Decision-Making in Multiagent Domains: A Weighted Constrained Potential Dynamic Game Approach imply that local solutions of this single problem are generalized Nash equilibria of the original game. That paper uses the reduction only for forward planning with known costs; it explicitly leaves inverse learning open. We supply the backward pass.
Research Program
Forward pass. The forward solver is an augmented-Lagrangian trajectory optimizer in the style of ALTRO. It runs once per interaction mode. A mode is a discrete ordering, such as "A merges ahead of B", and it is enforced by fixing which side of each pairwise separation constraint applies. Fixing the ordering makes each mode's problem locally well posed. A small classification head scores the modes, following the mode-scoring idea of Learning Game-Theoretic Models of Multiagent Trajectories Using Implicit Layers.
Backward pass. At the returned primal–dual pair $(y^*,\lambda^*,\nu^*)$, we linearize the KKT system restricted to the active constraints, as formalized for convex cones in Differentiable Convex Optimization Layers. The adjoint vector $v$ is obtained from
$$ \begin{pmatrix} \nabla^2_{yy}\mathcal{L} & A^\top \\ A & 0 \end{pmatrix} \begin{pmatrix} v_y \\ v_\lambda \end{pmatrix} = -\begin{pmatrix} \nabla_y \ell \\ 0 \end{pmatrix}, $$
where $\mathcal{L}$ is the Lagrangian of the potential problem, $A$ is the Jacobian of active constraints and dynamics, and $\ell$ is the training loss. Parameter gradients follow from contracting $v$ with the mixed derivatives in $\theta$.
The potential reduction matters here for a concrete reason. Differentiating a general game's equilibrium conditions gives a non-symmetric Jacobian. This system instead has the symmetric, time-banded structure of a single optimal-control problem. We can therefore reuse the parallel preconditioned-conjugate-gradient backward pass of Differentiable Model Predictive Control on the GPU. That method was built for single agents and replaces hard inequalities with soft barriers, so using it on multi-agent games with active-set constraints is an extension, not reuse as-is.
Mode switches. Weakly active constraints are those with zero multiplier sitting exactly on the boundary. These are where the yielding/overtaking switch happens, and there the classical theorem fails. We handle them in two steps. First, we detect them with a multiplier–slack threshold. Second, at those points only, we fall back to an interior-point smoothing with a small fixed barrier parameter. We report how often this fallback fires, because if it fires constantly the hard-constraint story weakens.
Analysis. We will study which weights are identifiable. When no shared constraint is active, the weights enter only through the private costs, and their overall scale is unidentifiable. Active separation constraints with nonzero multipliers are what pin down relative weights. The analysis predicts that recovery error should fall with the fraction of training scenes containing active interactions, and we test exactly that.
Baselines
The naive reference is the constant-velocity extrapolation used routinely in highway prediction, together with the black-box social-pooling regressors CS-LSTM and MFP. On highD, Learning Game-Theoretic Models of Multiagent Trajectories Using Implicit Layers reports 5.0 m and 5.2 m mean absolute error over a 7-second horizon for these two regressors. Any game model that fails to clearly beat them is not worth its solver.
The strongest existing method on highD is TGL-D from the same paper. It is an implicit layer through the stationarity conditions of a common-coupled potential game, with collisions as smooth penalties, and the paper reports 2.9 m mean absolute error at 7 s on highD. It is the most direct competitor because it differs from our method only in soft constraints, symmetric (unweighted) coupling, and an open-loop formulation. We will reimplement it on our encoder, solver tolerances, data splits, and tuning budget so that the constraint treatment is the only difference.
On Waymo Interactive the strongest competitor is the energy-based potential game layer of Energy-based Potential Games for Joint Motion Forecasting and Control. It solves several modes in parallel over a VectorNet encoder with a fixed number of warm-started optimization steps, and it reports a minSADE reduction of about 0.5 m over strong marginal baselines plus a lower overlap rate. The file gives no absolute number, so we will rerun it ourselves with an identical encoder and an equal GPU-hour budget.
Beating TGL-D means equal or lower 7 s error together with a zero overlap rate and better weight recovery on synthetic data. Beating the energy-based layer means matching its minSADE within noise while eliminating overlaps.
Three ablations isolate the key ingredient. The first, soft-penalty-weighted, keeps learned weights but turns constraints into penalties, with stiffness swept over two orders of magnitude. The second, hard-unweighted, keeps hard constraints with all $w_i$ fixed equal. The third, unrolled-hard, uses the same solver differentiated by unrolling instead of the implicit system.
Why not just…?
Why not just use a soft penalty with very high stiffness, as TGL and the energy-based layer already do? This is the real threat. A sufficiently stiff penalty approximates a hard constraint and avoids active-set bookkeeping. Our concern is that stiffness trades accuracy against feasibility and inflates the condition number of the Hessian the backward pass must solve against. It also absorbs part of the priority signal that the weights should carry. The stiffness sweep in the soft-penalty-weighted ablation settles this. If some stiffness matches our layer on error, overlap, weight recovery, and gradient stability at equal cost, the hard-constraint machinery is unjustified and we will say so.
Why not just unroll the augmented-Lagrangian solver? Unrolling stores every iterate and ties gradients to solver artifacts. Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond and Constraint-based graph network simulator document the memory cost. Differentiable Model Predictive Control on the GPU reports about 2x faster wall-clock convergence from implicit backward passes. Augmented-Lagrangian solvers also need outer multiplier loops, which deepens the unroll. The unrolled-hard ablation measures memory, gradient agreement against finite differences, and final accuracy at matched compute.
Why not just solve the general coupled game and differentiate that, without assuming a potential? General games capture interactions that are not symmetric, which our model cannot represent. However, Strategic Decision-Making in Multiagent Domains: A Weighted Constrained Potential Dynamic Game Approach reports forward solves of 26.15 ms for the potential reduction, against 248.21 ms for PATH and 577.35 ms for Algames. Those general solvers also yield non-symmetric backward systems. We will test the cost of the potential assumption directly: on synthetic games with a controlled non-symmetric interaction term, we will check where our layer's error departs from data generated by the true general game.
Why not just train a large transformer on WOMD? On raw displacement error with abundant data, a black-box model may well win, and we do not claim otherwise. Our claims concern feasibility, interpretable and recoverable priorities, and data efficiency. We therefore compare learning curves at 1%, 10%, and 100% of the training data, along with overlap rates, rather than peak accuracy alone.
Proposed Experiments
Synthetic recovery. We generate interactions from a known weighted constrained potential game, with randomized weights and margins and 2 to 6 agents in merges and bottlenecks. We measure relative-weight recovery error against the fraction of active-interaction scenes, and we expect recovery to improve sharply as that fraction rises. Soft-penalty-weighted should recover weights worse at every stiffness, or else the core claim fails.
Real data. We evaluate on highD and exiD with 7 s mean absolute error and final error, comparing against the reported 2.9 m for TGL-D. We also use the Highway Eagle Eye on-ramp set introduced in the TGL paper; no reported numeric result was found on Lacuna for it, so all methods there are our own runs. On the interactive subset of Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset we report minSADE, minFADE, overlap rate, and mode mAP.
Closed loop. In Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research we use the layer as a planner for the ego vehicle and report collision rate, off-road rate, and log displacement.
Decisive checks. We will check that learned yield behavior is predictable from multipliers. Specifically, the predicted share of separation burden, computed from multipliers and weights, should correlate with which vehicle actually decelerates in held-out merges. We will also report the fallback rate at weakly active constraints and the backward-pass latency against the forward solve.
If the hard-constrained layer fails to beat a well-tuned stiff penalty on weight recovery, the proposal reduces to an engineering variant of existing work, and we would report it as such.
References
- Strategic Decision-Making in Multiagent Domains: A Weighted Constrained Potential Dynamic Game Approach
- Learning Game-Theoretic Models of Multiagent Trajectories Using Implicit Layers
- Differentiable Convex Optimization Layers
- Differentiable Model Predictive Control on the GPU
- Energy-based Potential Games for Joint Motion Forecasting and Control
- Investigating Bi-Level Optimization for Learning and Vision From a Unified Perspective: A Survey and Beyond
- Constraint-based graph network simulator
- Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset
- Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research