Some generators make each sample by smoothly warping a fixed template image. A diffeomorphic warp, one that is smooth, invertible and never folds space, cannot add or remove holes or split one connected piece into two. So any training example whose shape is topologically different from the template is out of reach, however well the adversarial game is optimized. This note proposes a controlled study and a matching diagnostic for that situation. We train generative adversarial networks (GANs, a generator and a discriminator trained against each other) whose outputs are diffeomorphic warps of templates. On data whose topological classes we know in advance, we split observed mode collapse into two parts: modes lost to unstable minimax dynamics, which existing stabilizers should recover, and modes lost to topological unreachability, which no optimizer can recover. We then test whether a monitoring quantity used for GAN convergence, the proximal duality gap, can tell the two apart from the training run alone.
The question matters because two literatures pull in opposite directions. Medical-image generators increasingly warp verified anatomical templates in order to avoid hallucinated structure. Mode-collapse theory, meanwhile, says that continuous maps onto disconnected or non-convex targets must fail. Nobody has measured what happens when a GAN is forced to respect topology on data that does not. A practitioner who sees mode collapse in such a model currently cannot tell whether to tune the optimizer or change the representation. We aim to give that answer.
Core Claim
Two prior results frame the claim. Mode Collapse and Regularity of Optimal Transportation Maps argues from Monge–Ampère regularity that the transport map onto a disconnected or non-convex target is discontinuous, so continuous generators collapse onto one connected mode. Their remedy is to drop the adversarial game and fit a convex Brenier potential, the scalar function whose gradient gives the optimal transport map. Separately, stationary-velocity-field registration (Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces, VoxelMorph) guarantees a fold-free warp by integrating a smooth velocity field with scaling and squaring. That integration repeatedly composes a tiny displacement with itself.
These two results act on different maps, and this is the point the proposal turns on. The latent-to-velocity network $z \mapsto v(z)$ is an ordinary unconstrained network. The Lei et al. obstruction applies to it as it does to any GAN, and it is in principle curable by better optimization. The spatial warp $\exp(v(z))$ is a different matter. Each sample it produces is a diffeomorphic image of the template $T$. Its topology, meaning its connected components and holes, therefore equals that of $T$ up to interpolation error. Topology mismatch is a hard per-sample constraint, not an optimization failure.
We claim three things. First, in a single-template diffeomorphic GAN, mode coverage is bounded above by the topological class of the template. Second, gradient penalties, Hamiltonian updates and curvature nudging raise coverage up to that bound and no further. Third, when unreachable modes are present, training does not settle at a local stable stationary point, a saddle held in place by rotational dynamics as described in A Closer Look at the Optimization Landscapes of Generative Adversarial Networks. The discriminator keeps a nonzero gradient on the unreachable mass, so a persistent rotational component remains. The proximal duality gap should then plateau at a floor instead of decaying, and that floor serves as a signature of infeasibility.
Research Program
The generator maps a latent code $z$ and, optionally, a discrete template index $k$ to a velocity field $v$. It outputs $T_k \circ \exp(v)$, computed with a differentiable scaling-and-squaring layer as in VoxelMorph. The discriminator is a standard convolutional network on images.
We study three variants. The first uses one template. The second uses a fixed bank of templates, one per topological class, with a learned categorical selector. The third uses unconstrained displacement fields, which allow folding. Each is trained with four stabilizers. The first is plain simultaneous gradient descent–ascent. The second is the discriminator gradient-norm penalty from Gradient descent GAN optimization is locally stable. The third is Hamiltonian gradient descent from Last-Iterate Convergence Rates for Min-Max Optimization, which descends the squared norm of the game's vector field. The fourth is Hessian-aware nudging from Combating Mode Collapse in GAN training.
During training we log four diagnostics. The first is the top Hessian and game-Jacobian eigenvalues, estimated with stochastic Lanczos as Durall et al. and Berard et al. do; these give the rotational-versus-curvature balance. The second is the proximal duality gap $DG^\lambda$ of On Characterizing GAN Convergence Through Proximal Duality Gap, which those authors show tracks sample quality even at non-Nash attractors. The third is the minimum and distribution of $\det J_\phi$, the determinant of the spatial Jacobian of the warp. The fourth is per-class mode coverage measured with a pretrained classifier.
The precise prediction concerns the gap. Let $\mathcal{C}(T)$ be the set of target modes with the same topology as $T$, and let $\pi_{\text{out}}$ be the target probability mass outside $\mathcal{C}(T)$. Under a well-stabilized game, the single-template model's coverage should approach $|\mathcal{C}(T)|$, and $DG^\lambda$ should settle at a floor that grows with $\pi_{\text{out}}$. The pixel and multi-template models should show a floor near zero.
We also expect interpolation to provide leakage. Bilinear resampling can blur thin strokes and fake a topology change. When it does, the minimum of $\det J_\phi$ should collapse toward zero exactly on the samples that attempt unreachable modes. That gives a second, spatial signature tying the dynamics view to the deformation view.
The testbeds come from the file. MNIST and EMNIST digits fall into topological classes by hole count: 1, 2, 3, 5 and 7 have no holes; 0, 6 and 9 have one; 8 has two; 4 varies with handwriting. Stacked MNIST gives 1,000 combinatorial modes, so a single template per channel predicts a hard and computable ceiling on mode count. A synthetic shape set of disks, annuli and two-holed regions mixed in controlled proportions lets us vary $\pi_{\text{out}}$ directly. As a realistic sanity regime, we warp a brain template on OASIS T1 MRI from the Learn2Reg collection. Topology there is roughly constant, so the single-template model should hit no ceiling, and $DG^\lambda$ should decay as it does in pixel GANs.
Baselines
Do-nothing reference. This is the single-template diffeomorphic GAN trained with plain simultaneous gradient descent–ascent. Its mode count, eigenvalue trajectory and $DG^\lambda$ curve are the reference against which every stabilizer and representation change is measured. Beating it means recovering modes inside $\mathcal{C}(T)$ and showing $DG^\lambda$ settling at a stable value rather than oscillating.
Strongest existing stabilizer. This is the gradient-norm-penalized GAN of Gradient descent GAN optimization is locally stable. On Stacked MNIST it is the strongest result in the file: that paper reports its regularized GAN captured all 1,000 modes, while vanilla and unregularized Wasserstein GANs collapsed to a few. It is the right competitor because it is the cheapest method with a proof of local stability. If anything recovers topologically unreachable modes through dynamics alone, it should. We run it in pixel space, with the identical latent dimension, discriminator, training budget and penalty-weight sweep, to confirm the 1,000-mode result in our setup. We then run it with the single-template warp generator. Our hypothesis wins if the warp version stalls at the topological ceiling predicted from the template's class while the pixel version reaches about 1,000.
Other stabilizers. For curvature-based stabilization we include NuGAN from Durall et al. The file reports maximum Inception Score 8.46 versus 7.03 for the non-saturating GAN on MNIST, and mean Inception Score 8.53 versus 3.77 on EMNIST. Hamiltonian gradient descent is the rotation-removing extreme. Every stabilizer gets the same sweep budget, here 20 configurations each, selected on a held-out coverage metric.
Non-adversarial generator over warps. We add a diffusion model over velocity fields in the style of Diff-Def, which reports Dice 0.71 (age-conditioned) and 0.75 (ventricle-conditioned) on UK Biobank at 24 seconds per volume. Its role is to show whether the topological ceiling is independent of the training method.
Ablation of the key ingredient. This is the multi-template model with templates chosen by simple clustering, compared against the same model with a learned selector. It isolates whether the remedy needs anything beyond "one template per topology class."
Why not just…?
Why not just add a gradient penalty and call collapse solved? On pixel GANs this works: Nagarajan and Kolter report full 1,000-mode recovery on Stacked MNIST. Our hypothesis is that it cannot work for warp generators on topologically mixed data, because the missing modes are not reachable by any parameter setting. The penalty run on the single-template model is the direct test. If it reaches about 1,000 modes with $\det J_\phi$ bounded away from zero, our claim is false and we will report that.
Why not just use diffusion over deformation fields and skip the game? Diff-Def avoids minimax instability entirely. Its warps still come from one template, though, so our argument predicts the same topological ceiling for it. The diffusion baseline therefore checks whether the ceiling is independent of the training method. If diffusion with one template covers unreachable classes, the explanation must be interpolation leakage, and the $\det J_\phi$ logs will show whether that is the case.
Why not just drop warping, or the discriminator? Discriminator-free translation (DFMIR) reaches Dice 0.772 on Learn2Reg thorax–abdomen. It addresses registration, not sample diversity, so it does not answer our question. Pixel GANs cover modes but give up the anatomical guarantee that motivates warping. Generative Adversarial Networks for MR-CT Deformable Image Registration documents adversarially induced geometric hallucinations.
Why not just cluster the data and assign one template per cluster? This is a real threat to any method contribution. It may match a learned selector completely. If it does, we will present the work as a diagnostic result rather than a new method: the ceiling, the duality-gap floor and the vanishing $\det J_\phi$ signature, plus the recommendation to use topology-aware template banks. The clustering ablation settles the question.
Decisive Checks
Three outcomes decide the proposal. The first is mode count on Stacked MNIST and on the genus-controlled shape set. For the single-template warp GAN under each stabilizer, it should sit near the template-class ceiling and below the pixel-GAN count. The topology-matched multi-template model should close the gap. The second is whether the final value of $DG^\lambda$ rises monotonically with the unreachable fraction $\pi_{\text{out}}$ across the synthetic sweep. A clear rise means the gap diagnoses infeasibility. A flat profile means it does not. The third is whether, on OASIS, where topology is roughly constant, the single-template model shows no plateau. That would confirm the floor reflects topology and not the warp architecture itself.
We also report the full set of supporting measurements: fold fraction, compared against VoxelMorph-diff's reported average of 0.2 folding voxels; Inception Score on MNIST and EMNIST against the NuGAN figures; and top-eigenvalue trajectories. These distinguish rotational limit cycles from settling at a local stable stationary point.
References
- Mode Collapse and Regularity of Optimal Transportation Maps
- Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces
- VoxelMorph: A Learning Framework for Deformable Medical Image Registration.
- A Closer Look at the Optimization Landscapes of Generative Adversarial Networks
- Gradient descent GAN optimization is locally stable
- Last-Iterate Convergence Rates for Min-Max Optimization: Convergence of Hamiltonian Gradient Descent and Consensus Optimization
- Combating Mode Collapse in GAN Training: An Empirical Analysis using Hessian Eigenvalues
- On Characterizing GAN Convergence Through Proximal Duality Gap
- Learn2Reg: Comprehensive Multi-Task Medical Image Registration Challenge, Dataset and Evaluation in the Era of Deep Learning
- Diff-Def: Diffusion-Generated Deformation Fields for Conditional Atlases
- Unsupervised Multi-Modal Medical Image Registration via Discriminator-Free Image-to-Image Translation
- Generative Adversarial Networks for MR-CT Deformable Image Registration