Smoothing GAN Generators in Low Dimensions to Escape Mode Collapse

Generative adversarial networks (GANs) train two networks against each other: a generator that turns random noise into samples, and a discriminator that tries to tell those samples from real data. A common failure is mode collapse, where the generator settles on producing only a few kinds of output, for example a handful of digit classes, and ignores the rest of the data. We propose to treat collapse as a trap in a bumpy but roughly bowl-shaped loss surface, and to use a known result from optimization theory about when such traps can be escaped efficiently. That result shows that escape is cheap when the bumps are small relative to the square root of the number of directions you search in, and hopeless otherwise. Neural networks have millions of parameters, so smoothing the landscape in every direction at once is doomed. Smoothing only in a handful of directions where collapse actually happens may be enough.

Concretely, we would measure how far the generator's loss departs from a convex bowl along a few directions of high curvature at the moments training collapses. We would then replace the gradient in that small subspace with an average taken over a sphere of perturbations, with the sphere's radius set by the measured bump size rather than found by trial and error. In two sentences: collapse is a spurious local minimum inside a low-dimensional, nearly convex slice of the generator's loss, and averaging the loss over a small sphere in that slice, at a radius the theory prescribes, should remove the trap at modest cost. The idea is testable because it predicts both where smoothing helps (inside the slice, not everywhere) and how much smoothing is needed (a radius tied to the measured bump size).

Core Claim

The theory of approximately convex functions studies objectives $\tilde f$ that stay within an additive error $\Delta$ of some convex function. Algorithms and matching lower bounds for approximately-convex optimization shows that finding an $\epsilon$-approximate minimum in polynomial time is possible exactly when $\Delta = O(\max\{\epsilon^2/\sqrt{d}, \epsilon/d\})$ in dimension $d$. Its positive result is a zeroth-order method, meaning one that uses only function values, which averages the function over spheres. This bound is the constraint that shapes the proposal. With $d$ equal to the full parameter count, the tolerated $\Delta$ is essentially zero, which is why the same line of work, including simulated-annealing methods with $O^*(n^{4.5})$ oracle cost (Escaping the Local Minima via Simulated Annealing), has never touched deep networks.

On the GAN side, Combating Mode Collapse in GAN training: An Empirical Analysis using Hessian Eigenvalues found that collapse is preceded by spikes in the top few eigenvalues of the loss curvature. That suggests collapse is decided in a very low-dimensional part of parameter space.

Our claim joins the two observations. Restricted to a $k$-dimensional subspace spanned by the directions where collapse happens, with $k$ between 2 and 8, the generator's loss against a fixed discriminator is approximately convex, and its bump size $\Delta_k$ at collapse events falls within the tractable regime $\Delta_k \lesssim \epsilon^2/\sqrt{k}$. If that holds, sphere-averaging within the subspace at radius $r \propto \epsilon/\sqrt{k}$ removes the spurious minimum while barely biasing the objective. Isotropic smoothing over all parameters, at any radius, either fails to remove the trap or distorts the objective. The radius form follows the standard smoothing tradeoff: averaging a function with gradient bounded by $L$ over radius $r$ biases it by at most $Lr$, and it suppresses bumps narrower than about $r$. Balancing these two effects against the target accuracy $\epsilon$ gives the $\epsilon^2/\sqrt{k}$ tolerance.

Relation to Prior Work

The closest work is Greedy Adversarial Equilibrium. It smooths the minimizing player's objective by Gaussian convolution, gives the maximizing player Hessian-based greedy steps, and proves polynomial convergence for general nonconvex-nonconcave games. It smooths in the full space, and its exponents make it a proof of concept. It needs second derivatives for the maximizing player, does not tie its smoothing radius to the $\epsilon^2/\sqrt{d}$ boundary, and does not evaluate mode coverage. We keep the discriminator's ordinary first-order updates, smooth only the generator and only in a small subspace, set the radius from a measured $\Delta_k$, and judge success by modes recovered.

NuGAN motivates the choice of subspace. It also supplies the most direct competitor, because it projects gradients away from the top Hessian eigenvectors using repeated Lanczos iterations (an iterative method for estimating leading eigenvectors). We use the same directions but average over them rather than deleting them, and we estimate them from the recent history of generator gradients rather than from Hessian-vector products.

Convex Relaxation Regression fits convex surrogates to black-box functions and solved Salomon-type landscapes up to 100 dimensions where simulated annealing failed beyond 10. We reuse its convex fitting as the measurement tool for $\Delta_k$, which is feasible because $k$ is tiny.

Two further lines of work frame the problem differently. Mode Collapse and Regularity of Optimal Transportation Maps attributes collapse to the discontinuity of transport maps onto disconnected supports and sidesteps it with geometric optimal transport in an autoencoder latent space. We treat collapse instead as an optimization trap within the adversarial game, which is complementary. If their account were the whole story, subspace smoothing should not help on disconnected targets, and that is a check we run. The GAN is dead; long live the GAN! constrains the proposal. R3GAN's relativistic loss with two gradient penalties already recovers all 1,000 Stacked MNIST modes, with only local convergence guarantees. Any gain we claim must therefore appear in regimes where that baseline actually fails.

Research Program

The method has three components. The first is subspace discovery. Every $T$ steps we take the top-$k$ principal directions of the last few hundred generator gradients, stored as an orthonormal matrix $P \in \mathbb{R}^{p \times k}$, where $p$ is the parameter count. We check these against Lanczos eigenvectors on small models to verify that the cheap proxy tracks the curvature spikes NuGAN reports.

The second is measuring approximate convexity. We freeze the discriminator at logged checkpoints before, during, and after collapse and evaluate the generator loss $L_G(\theta + Pv)$ on a grid of offsets $v$. We then fit a convex surrogate in the style of Convex Relaxation Regression and record $\Delta_k$ as the largest deviation from that fit. This gives the first empirical map of how nearly convex GAN generator losses are in low dimensions, and how that changes as collapse approaches.

The third is the hybrid update. The generator's gradient is kept outside the subspace, and the part inside it is replaced by the gradient of the sphere-smoothed loss

$$L_G^{r}(\theta) = \mathbb{E}_{u \sim S^{k-1}}\big[L_G(\theta + r P u)\big],$$

where $S^{k-1}$ is the unit sphere in $k$ dimensions and $r$ is the smoothing radius. By Stokes' theorem, as in the Risteski–Li algorithm, this gradient is estimated from $m$ forward passes:

$$\widehat{\nabla} = (I - PP^\top)\nabla L_G(\theta) + \frac{k}{r m}\sum_{j=1}^{m} L_G(\theta + r P u_j)\, P u_j,$$

with $u_j$ drawn uniformly from $S^{k-1}$. The radius is set to $r = c\,\epsilon/\sqrt{k}$, where $\epsilon$ is the tolerated loss gap. The constant $c$ is calibrated once, on the synthetic benchmark only, using the measured $\Delta_k$. A known confound is that the discriminator moves between steps, so the landscape being smoothed is not fixed. We handle this by smoothing against the current discriminator only, which matches the setup Greedy Adversarial Equilibrium analyzes for its min player, and we report how $\Delta_k$ drifts across discriminator updates.

Baselines

The do-nothing reference is the non-saturating GAN trained with Adam, the setting in which NuGAN reports MNIST maximum Inception Score of 7.03 and EMNIST mean Inception Score of 3.77. Inception Score is a standard measure of sample quality and diversity. On mode-coverage benchmarks this baseline defines the collapse we are trying to prevent.

The strongest existing method is R3GAN (The GAN is dead; long live the GAN!), which reports 1,000 of 1,000 modes on Stacked MNIST and FID 2.75 on FFHQ-256, where FID measures the distance between generated and real image statistics. It is the strongest competitor because it achieves full coverage with first-order training alone. We run our hybrid update on top of R3GAN's loss and architecture, not as a replacement, with identical data and iteration counts, and with wall-clock time matched by giving R3GAN extra iterations equal to our $m$ forward-pass overhead. Beating it means recovering more modes, or recovering them in fewer wall-clock hours, in stressed regimes where R3GAN falls below 1,000. One such regime is Stacked MNIST with a discriminator reduced to a quarter of its width, which we expect to stress coverage; we will confirm this rather than assume it. The other is a 100-mode imbalanced Gaussian grid. Matching R3GAN in regimes where it already saturates counts as no gain.

The curvature-based competitor is NuGAN, which reports MNIST IS 8.46 and EMNIST IS 8.53. We run it with the same $k$ and compare coverage per wall-clock hour, since the cost of the Lanczos iterations is part of the comparison.

The optimal-transport competitor is AE-OT, which captured all 25 modes of the 25-Gaussian grid. We run it on the synthetic grids only, where its geometric computations are tractable.

Two ablations isolate the key ingredient. The first applies isotropic full-space sphere smoothing with the same forward-pass budget, with its radius grid-searched over the same number of trials we spend calibrating $c$. The second uses the same subspace smoothing with a grid-searched radius instead of the calibrated one. The first ablation tests whether low dimensionality matters. The second tests whether the approximate-convexity calibration adds anything beyond tuning.

Why not just…?

Why not just use R3GAN's dual gradient penalties, which already recover every Stacked MNIST mode? This is the main threat. The file reports no failure case for R3GAN, and its guarantee is only local convergence near an equilibrium. If R3GAN recovers all modes in our stressed regimes too, the method has no practical value for modern losses and survives only as a diagnostic for older ones. We will report that outcome plainly. The decisive runs are the reduced-capacity and 100-mode imbalanced settings, where a local guarantee has the most room to fail.

Why not just inject tuned isotropic parameter noise, as in Langevin-style training? Noise added to all parameters is the obvious simple alternative, and Convex Optimization with Unbounded Nonconvex Oracles using Simulated Annealing shows Langevin dynamics handles approximately convex objectives in principle. The Risteski–Li bound predicts, however, that in $p$ dimensions any radius large enough to wash out a bump of size $\Delta$ biases the objective by far more than $\epsilon$. The isotropic-smoothing ablation, run with an equal tuning budget, tests this directly. If it matches our method, the low-dimensional claim is refuted.

Why not just use NuGAN's projection, which already acts on the same directions? Projecting a direction out stops the generator from moving along it, and that can freeze a partially collapsed state instead of escaping it. Averaging along the direction keeps the coarse downhill trend while ignoring the bumps. NuGAN also pays for Lanczos iterations at every step. Coverage-per-hour comparisons at equal $k$ will settle whether the difference matters.

Why not just tune the smoothing radius by grid search and skip the theory? If grid search finds the same radius at similar cost, the approximate-convexity link is only descriptive. Its value would be transfer: a constant $c$ calibrated on Gaussian grids should give good radii on Stacked MNIST and CIFAR-10 without retuning. The calibrated-versus-searched ablation measures exactly this.

Proposed Experiments

The testbeds are the 25-Gaussian grid used by AE-OT, a 100-mode imbalanced extension of it, Stacked MNIST with modes counted by a pretrained classifier (R3GAN reports 1,000 modes), MNIST and EMNIST with Inception Score to match NuGAN, and CIFAR-10 with FID. For CIFAR-10, the reference is the JKO-regularized MMD-GAN at FID 15.76, down from 49.39 without regularization, reported with about 8.6% time overhead in A Unifying View of Variational Generative Wasserstein Flows. No reported R3GAN CIFAR-10 FID was found on Lacuna, so we will train it ourselves. The metrics are modes recovered, the fraction of samples falling between modes, IS, FID, and wall-clock time, each over five seeds.

The first decisive check tests the premise. Across logged collapse events, $\Delta_k$ in the gradient-history subspace should be small relative to $\epsilon^2/\sqrt{k}$. The same measurement along $k$ random directions, and the implied full-space value, should be far above it. If collapse-time landscapes are not approximately convex even in low dimensions, the theory does not apply and we stop there.

The second check tests the mechanism. With forward passes held equal, subspace smoothing should beat isotropic smoothing on mode coverage. Coverage as a function of radius should peak near the value the calibration predicts, not at an arbitrary one.

The third check tests practical value. On top of R3GAN in stressed regimes, the hybrid should raise coverage above the baseline under matched wall-clock time, at overhead comparable to the JKO method's 8.6%.

A fourth check addresses the competing explanation. On targets with widely separated, disconnected supports, if subspace smoothing still cannot eliminate samples between modes while AE-OT can, we conclude that transport-map discontinuity, not optimization traps, drives collapse there. We would then report which mechanism dominates in which regime.