Heart Chambers as Players: Best-Response Fitting of Coupled Shape Trajectories

Cardiac imaging pipelines usually model one structure at a time. A left-ventricle shape model, a right-ventricle shape model and a myocardium tracker are each fit on their own, and nothing stops the fitted surfaces from passing through one another or drifting apart over the heartbeat. Adjacent chambers share walls and push against each other, so these errors are physically meaningless, yet they persist because joint models need every structure annotated in correspondence on the same patients. We propose to treat each anatomical structure as a separate "player." Each player owns a compact statistical shape model, meaning a low-dimensional space of plausible surfaces learned from a population, and fits its whole trajectory across the cardiac cycle in one batch. The players then take turns: each re-fits its own full-cycle trajectory while the others are held fixed, paying a shared penalty whenever surfaces overlap. This turn-taking is called iterative best response.

Because the overlap penalty is symmetric between each pair of players, the turn-taking is coordinate descent on a single global objective. It therefore converges, and it only ever needs each structure's own fitting routine. This yields the main practical benefit: shape models built separately, on different cohorts or even with different model families, can be combined at test time into a contact-consistent multi-structure reconstruction without retraining anything jointly. The most uncertain parts are two empirical questions. The first is whether this modular scheme loses accuracy compared with optimizing everything jointly. The second is whether the order in which players move, which the sequential-decision literature shows strongly affects convergence, can be chosen to help poorly imaged structures.

Core Claim

Multi-structure cardiac motion tracking can be posed as an exact potential game. In a potential game, every player's private cost differs from one shared global function only by terms the player cannot change, so whenever one player improves its own cost it also improves the shared function. Here the players are anatomical structures and their strategies are whole-cycle trajectories of shape-model coefficients. Write $\alpha_k \in \mathbb{R}^{m \times T}$ for the coefficients of structure $k$ over $T$ frames. Write $D_k$ for its image-fit term, $R_k$ for its private temporal-smoothness term, and $C_{kl}$ for a symmetric penalty on interpenetration between structures $k$ and $l$. Player $k$ minimizes $J_k = D_k(\alpha_k) + R_k(\alpha_k) + \sum_{l \ne k} C_{kl}(\alpha_k, \alpha_l)$, and every such update decreases

$$ \Phi(\alpha_1,\dots,\alpha_K) = \sum_{k} \big[ D_k(\alpha_k) + R_k(\alpha_k) \big] + \sum_{k<l} C_{kl}(\alpha_k,\alpha_l). $$

This resolves the tension the literature raises between global temporal coupling and best-response convergence. Temporal coupling that lives inside a player's own term $R_k$ never breaks the potential structure. Only asymmetric cross-player terms do, for instance a rule that one chamber "yields" more than the other. A per-pair rescaling of such asymmetric penalties gives a weighted potential game, which keeps the convergence guarantee. The claim to be tested has two parts. Best response over per-structure batch fits matches the accuracy of joint optimization of $\Phi$ when joint optimization is possible. It also remains valid in the modular setting where joint optimization is not possible.

Relation to Prior Work

The mechanism is borrowed from GTP-SLAM. That paper casts multi-vehicle trajectory estimation with symmetric collision costs as a potential game and solves it by iterative best response, each vehicle re-fitting its whole trajectory. Its players, however, are rigid Dubins vehicles described by position and heading. We replace each vehicle with a deforming surface parameterized by a shape model, and the collision cost with a surface-overlap cost. The discrete precedent, JESP, shows that holding peers fixed turns an intractable joint problem into tractable single-agent ones. It also warns that best response stalls at local equilibria and needs restarts. That warning constrains our design and motivates the initialization stage below.

The per-player objective comes from global temporal optimization. AcinoSet's full trajectory estimation optimizes a rigid skeleton over a whole sequence and nearly halves error under occlusion compared with triangulation. Papadakis and Mémin optimize a single level-set curve non-causally using forward and backward (adjoint) passes. Neither uses a population shape model or more than one deforming body. We keep their non-causal, whole-sequence fitting and place it inside each player.

The parameterization follows statistical shape modeling. Adams et al. build a spatiotemporal shape model of a single left atrium with linear dynamics over 25 frames. Morgan et al. show that 18 modes explain 99% of right-ventricular variability. BrainODE rolls shape trajectories forward causally in a 150-dimensional principal-component space. All three model one structure. MulViMotion tracks the left-ventricular myocardium with a centralized network and weak shape supervision, but it does not model neighboring chambers. Our contribution is the coupling between structures.

Sequential action decomposition plays a supporting role in two places. The first is initialization. Operator decomposition turns a joint move over $N$ agents into $N$ conditional single-agent moves and prunes infeasible partial assignments early. We apply the same idea to choosing starting hypotheses: each structure has a few candidate trajectory basins, and we assign them structure by structure, discarding any combination that interpenetrates before the remaining structures are enumerated. The second is the turn order of the best-response sweep. The Multi-Agent Transformer notes that the order in which agents decide affects sample efficiency. We test whether a confidence-ordered sweep converges to better equilibria for weakly observed structures. In this sweep, well-contrasted structures move first and poorly imaged ones are fit against them.

Research Program

We build separate principal-component shape models for the left-ventricular blood pool, the myocardium and the right ventricle. Each model is built from its own training annotations, with no cross-structure correspondence. Each player's best-response oracle fits its coefficient trajectory over the full cycle by Gauss-Newton optimization. The fit uses a boundary-distance data term against a per-frame segmentation probability map and a second-difference smoothness prior; the variable count is small, roughly $m \times T$ at 20 modes and 25 frames. Contact enters through a symmetric, differentiable overlap penalty. This penalty is computed from signed distances of each surface's vertices to the other surface, with the myocardium–ventricle pairs encouraged to share a boundary rather than merely avoid overlap.

The initialization stage generates a few starting trajectories per structure from different mode basins. It assigns them sequentially with collision pruning, then hands the surviving combinations to the best-response sweep. We will establish monotone decrease of $\Phi$ formally. We will also check numerically, by measuring the sweep's behaviour, whether introducing a deliberately asymmetric contact penalty produces the cycling predicted for non-potential games.

The modular claim is tested by swapping one player's oracle for a heterogeneous one. The left-ventricular myocardium player is replaced by a black-box network tracker in the style of MulViMotion. That player's best response becomes a minimal correction of the network's output trajectory: the correction stays close to the network's prediction while paying the contact cost. This oracle has no shape model in common with the other players, so joint optimization of $\Phi$ is not available to any baseline.

Baselines

The naive reference fits each structure's shape model independently, frame by frame, with no temporal prior and no contact term. It shows the cost of ignoring both kinds of coupling. A second reference adds the batch temporal prior per structure but still no contact term. Comparing it with the full method isolates what the coupling adds beyond whole-sequence fitting, the gain that the AcinoSet comparison already suggests.

The ablation that isolates the key ingredient is joint optimization of the identical potential $\Phi$, run with L-BFGS and Gauss-Newton over all players' coefficients at once. It uses the same shape models, segmentation maps, initializations and wall-clock budget, with step-size and penalty weights tuned on the same validation split. If best response reaches a lower $\Phi$, or equal $\Phi$ at lower cost, the decomposition earns its place in the homogeneous setting. If joint optimization is better, best response is justified only by modularity. A second ablation compares random, fixed and confidence-ordered sweeps, and compares sequential pruned initialization with sampling joint hypotheses at the same evaluation budget.

The strongest existing method on the shared benchmark is MulViMotion on UK Biobank. It reports Dice 0.82, Hausdorff distance 14.59 mm and volume difference 8.62% for left-ventricular myocardium (MulViMotion). We will evaluate on its protocol where access permits. Beating it means matching its Dice within noise while reducing Hausdorff distance and incompressibility error, and reporting near-zero interpenetration with the right ventricle, which MulViMotion does not model. On ACDC, the strongest temporal competitor is the bidirectional ConvLSTM Res U-Net of Myocardial Segmentation with Temporal Consistency. That paper reports only a 1–2% Dice improvement over its baseline, up to 2.39% for some structures and phases; no reported absolute per-structure ACDC Dice for it was found on Lacuna. We will retrain it on the same split and use its probability maps as the data term for every shape-model method. All methods then share the same image evidence.

Why not just…?

Why not just optimize the joint objective directly? This is the main threat. With three structures, about 20 modes and 25 frames, $\Phi$ has roughly 1,500 variables, so joint optimization is cheap. The decomposition is not needed for tractability, as it was in JESP. The joint-$\Phi$ ablation settles the homogeneous case directly. We expect parity there, and we would report parity honestly. The case for best response then rests on the heterogeneous-oracle experiment, where no joint gradient exists. It also rests partly on the possibility that structure-by-structure updates escape the overlap-penalty local minima that trap joint descent; the restart statistics will show whether they do.

Why not just build one joint shape model over all chambers? Concatenating the left ventricle, myocardium and right ventricle into one principal-component model captures their relative placement statistically and needs no contact term. ACDC provides all three labels on the same patients, so we will build this model and compare it there on equal terms. The research file identifies two weaknesses. Linear combinations of coupled modes can still interpenetrate, and joint models require multi-structure correspondence on the same subjects. Joint correspondence rules out reusing structure-specific cohorts such as the tricuspid-regurgitation right ventricles of Morgan et al. or the atrial models of Adams et al.. If the joint model matches our accuracy and reports no overlap on ACDC, the contribution narrows to cross-cohort composition. We would state that result plainly.

Why not just train a larger end-to-end network for all chambers? A centralized network could output all structures at once. In the file, the plain 3D U-Net in MulViMotion trails the shape-aware model (Hausdorff 17.47 mm versus 14.59 mm), and no paper in the file shows end-to-end networks enforcing contact. We include the retrained ConvLSTM network both as a competitor and as the evidence source. Any gain we show therefore comes from the coupling, not from better pixel classification.

Proposed Experiments

The primary benchmark is ACDC, with left ventricle, right ventricle and myocardium labels across its five pathology groups on the standard 70/30 split. We report Dice, Hausdorff distance, interpenetration volume between structures, the frame-to-frame variation of the volume curve, and runtime. Results are stratified by pathology, because dilated right ventricles should benefit most from being fit against a well-imaged left ventricle. Following the volume-difference protocol of MulViMotion, UK Biobank tests whether adding the right-ventricle player harms left-ventricular accuracy.

The cross-cohort test is decisive for the modularity claim. A right-ventricle shape model trained only on one group of ACDC patients and a left-ventricle model trained only on a disjoint group are composed on held-out patients. The composition is then compared against the joint shape model trained on overlapping subjects. A further variant swaps in the black-box myocardium tracker as one player.

Three outcomes would settle the proposal. First, best response should reach the joint optimizer's $\Phi$ within a small tolerance across seeds. Second, the confidence-ordered sweep should give lower right-ventricle Hausdorff distance than random order, with the gap largest in the abnormal-right-ventricle group. Third, the cross-cohort composition should reduce interpenetration to near zero without losing Dice relative to the per-structure batch fits. If the first holds and the third fails, the method is no better than a joint optimizer. If both hold, separately trained anatomical models can be combined into a physically consistent beating heart.