Offline model-based optimization asks a deceptively simple question: given only a static dataset of designs and their measured scores — DNA sequences, molecules, robot morphologies — find a new design that scores higher than anything in the dataset. The standard loop trains a surrogate scorer on the dataset and then searches for inputs that maximize it. This loop fails in two opposite ways. Search confined to the data's neighborhood collapses prematurely into a local optimum; search that wanders off the data manifold exploits the surrogate's blind spots and "discovers" designs whose measured scores are garbage. DynAMO (Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization) addresses the second failure with an adversarial critic — a classifier trained to distinguish real dataset designs from generated candidates — that penalizes leaving the data distribution. But a single critic saturates: after a few rounds it labels every generated candidate as fake with near-total confidence, its decision surface goes flat, and the generator receives a verdict ("all invalid") with no direction attached.
The obvious repair is to replace one critic with several, so that some critic is always close enough to being fooled to pass gradient. The trouble is that critics trained on the same data with the same objective make the same mistakes: Quantifying Correlations of Machine Learning Models shows that error correlations between such models routinely exceed 0.7, so a naive critic ensemble saturates in unison — the instrument meant to detect exhaustion of the search region is itself broken by redundancy. This proposal makes that failure the signal. We treat a critic ensemble as a measurement device: where critics disagree about a candidate, gradient still exists; where they agree, a region of the search space is exhausted; and when their errors correlate, the measurement itself has collapsed. A closed-loop controller reads these three quantities and does three things — re-initializes the most redundant critic, expands the active search region along disagreement directions, and contracts it when median candidate quality falls — turning both ensemble diversity and search-space growth from hand-set heuristics into monitored, data-driven variables.
Core Claim
The claim is that in adversarial offline design, the critic ensemble's output agreement and measured error correlation form a sufficient closed-loop signal for two decisions that current methods make by schedule or by heuristic: when to expand the search space and how to keep the critic ensemble useful. Concretely, at step $t$ we track $s_t$, the fraction of generated candidates that all $K$ critics classify as out-of-distribution with confidence above $1-\epsilon$ (a saturation-and-agreement signal), and $\rho_t$, the mean pairwise error correlation between critics on a held-out probe set, estimated with the Pearson and $\phi_K$ machinery of Quantifying Correlations of Machine Learning Models. The active expansion radius $r_t$ around the current proposal cloud follows
$$r_{t+1} = r_t \cdot \exp\!\big(\eta\,(s_t - s^\*)\big),$$
gated by two conditions: expansion fires only when $\rho_t$ is below the redundancy threshold (motivated by the 0.7 correlations reported empirically), and contraction overrides expansion whenever the median score of proposed candidates drops below a lower quantile of the offline dataset. The direction of expansion is likewise read off the instrument: we expand anisotropically along the principal directions of the candidates on which critics disagree most, since disagreement is exactly where the softmax-aggregated generator loss retains nonzero gradient.
This differs from the closest existing work on every axis at once. GMAN (Generative Multi-Adversarial Networks) also pits a generator against a critic ensemble with a "forgiving teacher" softmax aggregation, but it maintains critic diversity only through heuristic dropout and depth perturbations, never measures whether the critics have become redundant, and operates in a fixed coordinate space. DynAMO uses a single critic as a static manifold constraint — it cannot expand into unobserved regions at all, and its diversity mechanism costs roughly 180% extra runtime while degrading median batch quality. GrowTAS (GrowTAS: Progressive Expansion from Small to Large Subnets for Efficient ViT Architecture Search) does expand the search space progressively, but on a hardcoded transition epoch $T_1$ rather than a closed-loop signal, and without any critic. Our proposal is the combination none of them makes: correlation-monitored critics whose agreement times the expansion, whose redundancy triggers re-initialization, and whose disagreement steers its direction.
Research Program
Three components get built. First, the instrumented ensemble: $K$ critics, aggregated by the forgiving-teacher weighting inherited from GMAN but with each critic's weight additionally discounted in proportion to its measured error correlation with the ensemble consensus, so a redundant critic cannot dominate the generator's gradient even before it is re-initialized. Re-initialization is from scratch rather than fine-tuning, because Neyshabur et al. show that models sharing initial weights remain trapped in the same optimization basin — perturbing a redundant critic would leave it correlated. Second, the expansion operator: nested active regions $A_1 \subset A_2 \subset \dots$ in the spirit of GrowTAS, but with transitions triggered by the saturation-agreement gate rather than an epoch count, and with the growth concentrated along disagreement directions to respect the "thin manifold" problem that Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization documents — high-scoring designs live on thin, low-dimensional manifolds, so isotropic expansion would dilute sampling into unviable regions. Third, the contraction guard, which addresses DynAMO's own acknowledged weakness that diversity-seeking degrades the median candidate: expansion is rolled back when it stops paying for itself in median quality. The contraction guard also protects against the unimodal penalty documented for Divergence-Guided Particle Swarm Optimization, whose anti-convergence repulsion actively hurt on the smooth Sphere function — on a smooth surrogate task our gate should read high agreement everywhere and contract to plain exploitation, paying little for the machinery.
A small theoretical piece accompanies the mechanism: when at least one critic is not saturated on a candidate, the softmax-weighted generator loss has nonzero gradient there, so the disagreement directions are provably the directions along which expansion can still teach the generator something. This is the formal bridge between discriminator saturation and dynamic search-space expansion — the ensemble's disagreement pattern is a map of where the minimax game still has gradients, and expansion should follow the map.
Baselines
The naive reference point is whitened gradient ascent on the surrogate, with inputs and outputs scaled to unit normal statistics. This is not a strawman: the Design-Bench paper (Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization) reports that this naive method and CMA-ES are competitive with — and often beat — sophisticated generative models in 4 out of 8 tasks, so any proposal must clear a bar that simple search already sets. We run it with identical surrogate training and the same proposal budget as every other method.
The strongest competitor is DynAMO (Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization), the existing method that combines an adversarial in-distribution critic with diversity-seeking distribution matching; it is the direct predecessor whose two open problems (single-critic saturation, static search bounds) our proposal targets. It will be run on the same six Design-Bench tasks (TFBind8, TFBind10, Superconductor, ChEMBL, D'Kitty, Hopper), with the same surrogate architecture, the same number of tuning configurations searched, and the same wall-clock allowance. The numbers to beat, taken from that paper: an average rank of 1.2 across optimizers on Best@128 (where 1 is best), an improvement of the pairwise-diversity optimality gap from $-53.2$ to $+27.8$ when paired with gradient ascent, and an optimality gap of 74.2 under BO-qUCB. Because DynAMO's own weaknesses are median-quality degradation and roughly 180% runtime overhead, beating it means matching or improving its average rank while raising median batch reward above exploitation-heavy baselines and cutting overhead below its 180%. We also include COMs, which the Design-Bench paper identifies as a top performer at the 100th percentile (best-design-found) across tasks, particularly on Hopper; no reported result found on Lacuna for COMs' per-task scores, so we will reproduce it under our budget.
The key ablation isolates the closed-loop gate itself: the same $K$-critic ensemble with GMAN-style heuristic diversity (varying dropout and depth, forgiving-teacher aggregation) but no correlation monitoring and no expansion — plus two further variants, decorrelation-only (no expansion) and expansion-only (heuristic diversity, gate drives timing but critics are never re-initialized), and an expansion variant triggered by a fixed GrowTAS-style schedule instead of the saturation signal. If the gated version does not beat all of these, the correlation-and-agreement signal is not earning its keep.
Why not just…?
Why not just use GMAN's forgiving teachers? GMAN already fixes vanishing gradients by aggregating over $N$ critics with a softmax that favors the most fooled ones, and its adaptive $\lambda$ annealing is a crude closed loop of its own. But GMAN's diversity rests on dropout and depth perturbations, and the correlation evidence says that is not enough: same-data, same-objective models drift past 0.7 error correlation (Quantifying Correlations of Machine Learning Models), which means the whole ensemble can saturate together and the forgiving teacher has nothing left to forgive. Our first experiment directly tests this: we instrument a GMAN-style ensemble on our tasks and plot $\rho_t$ over training. If the correlation trace stays low, GMAN's heuristic really does suffice in this regime and the proposal's decorrelation machinery is unnecessary — that would be a decisive negative result, and we will report it. We expect it to climb, and additionally GMAN has no notion of search-space expansion at all, so it cannot address DynAMO's manifold confinement.
Why not just re-initialize critics on a fixed schedule? This is the cheapest version of our idea and a real threat. We will run it as an ablation, and if it matches the full method on Design-Bench, the contribution narrows honestly to the expansion-direction rule. We think it will fall short for two concrete reasons. Scheduled restarts re-initialize critics that may still be informative while leaving redundant ones in place, wasting the fixed critic budget; the correlation monitor targets re-initialization where it matters. More importantly, restarts answer when to refresh the instrument but not where to search next — they provide no signal for expansion timing or direction, so a restart-only method must still expand by schedule or isotropically, inheriting both the GrowTAS rigidity problem and the thin-manifold dilution problem.
Why not just use a non-saturating critic loss (Wasserstein or gradient penalties)? Loss surgery keeps gradients alive in principle, and Some Theoretical Insights into Wasserstein GANs shows game stability can be tuned through architecture ratios. But in this setting the critic's job is not merely to stay informative — it is supposed to penalize out-of-distribution candidates to prevent reward hacking. A critic that never saturates anywhere is a critic that no longer constrains the search to the data manifold, which reinstates the exact failure DynAMO was built to stop. We will run a Wasserstein-critic variant of DynAMO as a baseline to check how much protection it sacrifices; our disagreement-gated expansion is the selective alternative — it relaxes the manifold constraint only along directions where critics still struggle, rather than globally.
Proposed Experiments and Decisive Checks
The primary evaluation runs on the Design-Bench suite (Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization) across the six tasks used by DynAMO: TFBind8, TFBind10, Superconductor, ChEMBL, D'Kitty, and the 5,126-weight Hopper controller. We report Best@128 (score of the best design among 128 final proposals), median batch reward (the honest metric for diversity-seeking methods, since DynAMO's own results show median degradation), pairwise diversity, and wall-clock overhead relative to the underlying optimizer. The decisive checks, in order of how much they would surprise us if they failed: first, the correlation trace — a GMAN-style ensemble's $\rho_t$ should climb toward and past 0.7 during training while the monitored ensemble holds it below threshold; second, the gradient-norm trace — the decorrelated ensemble should keep the generator's received gradient nonzero exactly on the disagreement subspace, and expansion along those directions should raise median reward relative to isotropic expansion; third, the schedule comparison — the saturation-gated transitions should beat a fixed-epoch GrowTAS-style schedule and a random-timing schedule at matched expansion budgets; fourth, the unimodal check — on a smooth surrogate task the gate should contract to exploitation, avoiding the unimodal penalty that repulsion-based DPSO pays on Sphere; and fifth, overhead — sharing the probe set across critics and re-initializing rarely should land well below DynAMO's 180% figure. A secondary, smaller study isolates the decorrelation ingredient alone in a setting with no search-space dynamics: standard image GAN training on MNIST and CIFAR-10, where GMAN reports an Inception Score of 5.878 on CIFAR-10 and MAGAN (MAGAN: Margin Adaptation for Generative Adversarial Networks) reports 6.40 against Wasserstein GAN's 3.82 — if correlation-discounted aggregation beats heuristic diversity there too, the ensemble-redundancy fix generalizes beyond offline design. As a stretch extension, the same gating applies to sequence design with uncertainty ensembles in the style of Biological Sequence Design with GFlowNets, which reported a performance score of 0.932 on antimicrobial peptide design with deep-ensemble uncertainty — regression ensembles suffer the same redundancy, and their disagreement could gate sequence-space expansion, though we treat this as follow-up rather than a promised result.
References
- Diversity By Design: Leveraging Distribution Matching for Offline Model-Based Optimization
- Quantifying Correlations of Machine Learning Models
- Generative Multi-Adversarial Networks
- GrowTAS: Progressive Expansion from Small to Large Subnets for Efficient ViT Architecture Search
- What is being transferred in transfer learning?
- Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization
- Divergence-Guided Particle Swarm Optimization
- Some Theoretical Insights into Wasserstein GANs
- MAGAN: Margin Adaptation for Generative Adversarial Networks.
- Biological Sequence Design with GFlowNets