Holes or Thin Spots? Using MAP-Elites to Diagnose and Repair Mode Dropping in GANs

A generative adversarial network (GAN) trains a generator that turns random input vectors into samples, and a discriminator that tries to tell those samples from real data. When the generator stops producing some kinds of data, the problem is called mode collapse, or mode dropping. It is usually reported as a single number, such as "recovered 900 of 1,000 modes." That number mixes two different failures. In the first, a missing mode is a hole: no input vector produces it, because the generator never learned it. In the second, the mode is a thin spot: some input vectors do produce it, but so few of them that ordinary random sampling almost never lands there. These failures need different fixes. A hole requires changing the generator's weights. A thin spot only requires changing how input vectors are drawn, with no retraining at all.

We propose using MAP-Elites to tell the two failures apart. MAP-Elites is a search method that keeps one best solution in each cell of a grid laid over chosen output properties. Here, each cell is a region of a frozen feature space that real data occupies. We search the generator's input space and record which data-occupied cells are reachable. Reachable but under-sampled cells are repaired by replacing the generator's sampling distribution with a mixture built from the search archive, weighted by how often real data falls in each cell. Cells that remain empty after a large search budget are treated as holes. These are repaired during continued training, using targeted gradient steps and discriminator batches drawn from the missing region. The core claim is that, in modern well-regularized GANs, much of the remaining distribution mismatch consists of thin spots rather than holes. If so, an archive-based correction to the sampling distribution should reduce mode-frequency error at a cost far below further adversarial training.

Core Claim and What Is New

Two bodies of work sit at this intersection, and neither does this.

The first runs quality-diversity search in a GAN's input space, but only after training and only for control. Illuminating Mario Scenes in the Latent Space of a Generative Adversarial Network runs MAP-Elites variants over a frozen DCGAN to find playable levels with chosen properties. Differentiable Quality Diversity introduces CMA-MEGA, which backpropagates objective and descriptor gradients into a frozen StyleGAN's latent space and reports reaching full archive coverage where black-box MAP-Elites stays below 2%. Both papers treat the generator's coverage as a fixed limit. Neither asks whether that limit reflects missing modes or merely a badly weighted sampler. We reuse their search machinery, CMA-MEGA in particular, but use it as a measuring instrument and as a source for a corrected sampling distribution.

The second body of work fights collapse during training with multiple critics or evolving populations. Dual Discriminator Generative Adversarial Nets, DuelGAN, Cost Ensemble with Gradient Selecting for GANs and CDE-GAN all change the training game itself. None of them separates holes from thin spots, so each pays training-time cost even for failures a cheap sampling fix would resolve. CDE-GAN, for example, explicitly reports higher per-iteration cost.

The obvious combination would be a MAP-Elites archive of discriminators or generators. We deliberately do not propose that. The GAN is dead; long live the GAN! constrains the design space. Its R3GAN uses one generator and one discriminator, a relativistic paired loss (which scores real and fake samples relative to each other) and gradient penalties on both. It recovers all 1,000 Stacked MNIST modes and reaches FID 2.75 on FFHQ-256. That makes population-based training a weak bet against the strongest single-critic baseline. Our proposal is designed to sit on top of R3GAN rather than compete with it.

Research Program

The mechanism has three parts.

The first is the descriptor grid. On Stacked MNIST, the grid is literal. Each image stacks three MNIST digits into the color channels, so a frozen digit classifier applied to each channel gives a 10×10×10 MAP-Elites grid whose cells are exactly the 1,000 modes. On CIFAR-10 and FFHQ, we embed real images with a frozen feature extractor and partition the embeddings into a few thousand Voronoi cells, using a centroidal Voronoi tessellation (CVT) fitted to real data. We record the real-data cell frequencies $\pi_{\text{data}}(c)$, the fraction of real images falling in cell $c$.

The second part is reachability search. For a trained generator $G$, we run CMA-MEGA over the input vector $z$. The objective is the discriminator's realism score, which keeps the search on the data-like part of the generator's output. The descriptors are the cell-embedding coordinates. Each cell keeps up to $k$ input vectors that land in it, rather than a single elite. Gradient-Informed Quality Diversity for the Illumination of Discrete Spaces reports that gradient-driven search against a frozen embedding model finds adversarial-looking outputs, so we guard against this directly. A cell counts as reached only if a second, independently trained feature extractor agrees on the cell assignment, and only if the output passes the realism score. We report reachability as a curve over search budget, not a single verdict. "Unreached at budget $B$" is evidence of a hole, not proof of one.

The third part is the repair, which differs by failure type. For thin spots, we replace the standard Gaussian sampler with

$$p'(z) = \sum_c \pi_{\text{data}}(c)\, q_c(z),$$

Here $q_c$ is a small Gaussian fitted to the archived input vectors for cell $c$, and the sum runs over reached cells. Sampling from $p'$ matches the cell frequencies of real data by construction, up to how well each $q_c$ keeps its samples inside its own cell, which we measure. For holes, we resume training with an added term. Input vectors are taken from the archived elites nearest the empty cell and pushed by descriptor gradients toward that cell. In the same steps, the discriminator's real batches are enriched with real images from that cell. After each repair round, search is re-run and $p'$ is refit.

The analysis question running through the program is how mode error splits between holes and thin spots. We measure this across generators of increasing quality, from DCGAN through StyleGAN2 to R3GAN, and across checkpoints during training.

Baselines

The naive reference is the unmodified generator sampled from its standard Gaussian. Every metric is reported for it first. On FFHQ-256, R3GAN reaches FID 2.75 against StyleGAN2's 3.78 (The GAN is dead; long live the GAN!).

The strongest existing method is R3GAN itself, trained with its published recipe. On Stacked MNIST it reports recovering all 1,000 modes, while StyleGAN2 "struggles to reach 900." Because R3GAN already saturates mode count, beating it requires a sharper metric than mode count. Our primary metric is the KL divergence between the generated and real mode-frequency histograms. We found no reported result on Lacuna for mode-frequency KL for R3GAN, so we will measure it ourselves. To be fair, every compute cost of our method is converted into additional R3GAN training steps of equal floating-point operations, and R3GAN receives the same hyperparameter tuning budget. This includes search passes and repair-phase steps. Beating R3GAN means a lower mode-frequency KL at equal compute, with FID no worse than R3GAN's within seed variance.

We also compare against the published coverage results of the multi-critic methods. On the 2D-ring benchmark, CES-GAN reports 24.8 ± 0.2 of 25 modes (Cost Ensemble with Gradient Selecting for GANs), and on CIFAR-10 it reports FID 19.7. On MNIST-1K, D2GAN reports 1000.0 ± 0.00 modes (Dual Discriminator Generative Adversarial Nets). We rerun D2GAN under our compute budget on the long-tailed benchmark described below.

Three ablations isolate the key ingredient. The first replaces archive-derived $q_c$ with input vectors collected by plain random sampling and binned by cell. This tests whether search adds anything over cell-binning of random samples. The second applies the hole-repair training step to randomly chosen cells instead of diagnosed holes. The third applies thin-spot resampling alone, without hole repair.

Why not just…?

Why not just use R3GAN and stop? On uniform Stacked MNIST, R3GAN may indeed leave nothing to fix, and we expect our method to match it there. The open question is whether R3GAN reproduces mode frequencies when modes are imbalanced, because mode count is blind to frequency. The long-tailed benchmark settles this. If R3GAN's mode-frequency KL is already at the level expected from the finite number of evaluation samples, the thin-spot claim is false and we will report it as such.

Why not just apply rejection sampling with the discriminator, or with the cell classifier, after training? This is the real threat for thin spots, since it also corrects sample frequencies without retraining. Its weakness is cost. To produce a mode with prior probability $p$ under the generator's sampler, rejection sampling needs on the order of $1/p$ draws, whereas archive resampling draws input vectors directly from $q_c$. We will sweep the imbalance in the long-tailed benchmark. The prediction is that the two methods tie for moderately rare modes and that rejection sampling's cost per accepted sample grows sharply for the rarest. If they tie everywhere, the sampling half of the proposal reduces to a cheaper implementation of a known idea, and the contribution narrows to the diagnosis.

Why not just upweight rare real images during training? Inverse-frequency reweighting changes the target distribution and pays training cost for every mode. It cannot distinguish a mode the generator lacks from one it under-samples. We run it as a baseline matched to equal compute, measured both on hole-filling and on mode-frequency KL against the true data distribution. The prediction is that it fills holes but biases frequencies away from the data distribution.

Why not just run offline latent search as in the Mario and DQD work? That search is our reachability step. Used alone, as in Illuminating Mario Scenes in the Latent Space of a Generative Adversarial Network, it cannot recover true holes. The hole-repair ablation measures exactly what the in-loop step adds beyond search.

Proposed Experiments and Decisive Checks

The central benchmark is a long-tailed Stacked MNIST, which we construct by sampling digit triples with power-law frequencies at several exponents. Because ground-truth mode frequencies are known, holes and thin spots can be verified independently of our diagnosis. We supplement it with the 2D-ring and 2D-grid synthetic datasets, CIFAR-10, and FFHQ-256. All runs use three or more seeds.

The metrics are mode count, mode-frequency KL, reachability-versus-budget curves, FID, and the cell-agreement rate between the two independent feature extractors. The agreement rate serves as an adversarial-gaming check.

Three results decide the claim. The first is the decomposition itself. On long-tailed Stacked MNIST, for StyleGAN2 and R3GAN, what fraction of the excess mode-frequency KL disappears under archive resampling alone, with no retraining? The hypothesis predicts most of it. The second is whether the search-based diagnosis agrees with ground truth: cells our search labels as holes should be ones that remain absent even when sampling is heavily concentrated near the nearest elites. The third is whether hole repair fills diagnosed holes faster than both the random-cell ablation and equal-compute R3GAN training, without raising FID.

On CIFAR-10 and FFHQ there is no ground truth, so we report the decomposition only where the two feature extractors agree. A result in which holes dominate and rejection sampling matches archive resampling would refute the cheap-fix thesis and leave a narrower diagnostic contribution.