Evolving Differentiable Loss Code by Reading the Metric

Many of the scores we care about cannot be differentiated. PESQ (Perceptual Evaluation of Speech Quality) and STOI (Short-Time Objective Intelligibility) in speech, and the precision–recall break-even point in ranking, are procedural programs full of thresholds, alignments and lookups. Gradient descent cannot pass through them. The two standard workarounds both have known costs. One trains a neural network to imitate the metric and backpropagates through that imitation. This approximation drifts as the model being trained changes. The other optimizes a hand-designed smooth proxy, which often does not match the metric well. This note proposes a third route. A large language model (LLM) is given the metric's actual source code. It is then asked, inside an evolutionary loop, to write and repeatedly rewrite short differentiable Python loss functions. Candidates are kept or discarded according to a cheap test: does a small gradient step on this loss raise the true metric?

The key idea fits in two sentences. The LLM acts as a "semantic mutation and crossover" operator: instead of flipping random symbols, it reads two parent loss functions and the metric code, and writes a coherent new loss that combines or repairs them. Its fitness is measured by short gradient-step probes from saved training checkpoints, not by full training runs. The result is a readable, fixed loss function that tracks a non-differentiable metric. This targets a specific gap. Nobody in the material has used code-level LLM evolution to build training losses for neural networks under a non-differentiable metric. Nobody has used the metric's own procedure as context for writing the loss either.

Core Claim

The claim is that semantic evolution over loss code, conditioned on the metric's source and scored by one-step metric gain, finds losses that match learned-surrogate methods on a single metric. The claim is also that it beats them in the regime where learned surrogates are known to break, which is several conflicting metrics optimized at once. It should do this at a search cost far below evolutionary methods that need full training for each candidate.

The fitness of a candidate loss $L$ is defined over a set $C$ of saved checkpoints $\theta$ taken at different stages of a reference training run:

$$F(L) = \frac{1}{|C|}\sum_{\theta \in C} \Big[ M\big(\theta - \eta \, \tfrac{g_L(\theta)}{\|g_L(\theta)\|}\big) - M\big(\theta - \eta \, \tfrac{g_0(\theta)}{\|g_0(\theta)\|}\big) \Big]$$

In this definition, $M$ is the true metric on a held-out batch, $g_L$ is the gradient of the candidate loss summed over a few steps, $g_0$ is the gradient of a reference loss (plain mean squared error), and $\eta$ is a fixed step length. Normalizing the step length makes $F$ insensitive to the loss's overall scale. That matters because rescaling is the cheapest way for a candidate to look good. Spreading $C$ across training stages guards against a loss that helps early in training and hurts late.

Relation to Prior Work

Three works supply the method. Evolution of Heuristics evolves pairs of a natural-language "thought" and executable code using five LLM prompt strategies. It beats FunSearch on online bin packing (1.18% gap to the lower bound versus 2.23%) with about 2,000 queries. We reuse its explore and modify operators. Its targets, however, are standalone combinatorial rules that never drive gradient training. L-SFE shows that LLM mutation, crossover and "concept injection" on Python score functions can beat BIC for causal discovery under Structural Hamming Distance. Its score functions guide a discrete graph search, not a neural network. EvoPrompt contributes the differential-evolution crossover, in which the LLM states how two parents differ and applies that difference to the best one. We adapt it to code.

Two works motivate the proposal by exposing the gap. LossAgent puts an LLM in the training loop under non-differentiable image quality metrics. It reaches NIQE 4.08 against 4.23 for fixed weights and 4.28 for random search, but it only adjusts scalar weights over a fixed set of L1, VGG perceptual and adversarial losses. Our loss is open-ended code rather than a weighted sum. CSE-Autoloss evolves loss graphs from 21 math primitives for detection mAP and gains 1.1% on Faster R-CNN. It is the closest prior system. Its blind genetic operators produced so many broken formulas that over 99% had to be filtered, and the survivors are hard to read. We replace those operators with LLM operators that read the metric. We keep its idea of rejecting candidates with a cheap simulated training run, and our one-step probe is a sharper version of that filter.

Two works constrain the design. SPELL found that LLM optimizers do better when shown bounded scores such as accuracy than raw continuous loss values. So the LLM never sees raw loss magnitudes. It sees bounded diagnostics: relative metric gain over the reference, the fraction of checkpoints where the step helped, and per-component breakdowns of the metric. MetricGAN-OKD showed that gradients from separate learned surrogates for PESQ and STOI conflict. It resolved the conflict by distilling knowledge across multiple generators. That is the regime where a single composite loss, written with both metrics' code in view, has the most room to help.

Learned surrogates are the incumbent we must beat or match. These include Quality-Net, MetricGAN+ and CMGAN. Analytic relaxations play a similar role: PAMSE smooths images before MSE to track SSIM, and Mao, Mohri and Zhong derive principled surrogates for F-beta and the Jaccard index. Our method can be seen as automating what PAMSE's authors did by hand, namely reading a metric and writing a smooth loss that follows it.

Research Program

Each individual in the population is a triple. It contains a short thought ("weight errors in bands where the PESQ loudness model is most sensitive"), a Python function from model output and reference to a scalar loss, and the fitness defined above. Seed individuals are MSE, scale-invariant signal-to-noise ratio, and a one-shot LLM attempt at a differentiable PESQ.

The operators follow Evolution of Heuristics, with three additions:

  • Every prompt includes the relevant excerpt of the metric's reference implementation.
  • A "repair" mutation is used. The LLM sees the batches where the candidate's step lowered the metric and is asked to explain why and fix the loss.
  • An injection operator in the style of L-SFE proposes a new mechanism, such as a soft-alignment term or a smoothed threshold.

Before scoring, each candidate is screened automatically. It must compile, produce finite gradients, and give a nonzero gradient on random inputs. The few survivors of the probe stage then receive full training.

The analysis component asks what the evolved losses actually encode. We check which metric components the evolved code mirrors, and whether those components predict the gain. For the classification domain, we also check whether evolution rediscovers known principled surrogates.

Baselines

The naive reference is training the same generator with plain MSE. MetricGAN+ reports PESQ 2.71 for this on VoiceBank-DEMAND, and we re-run it under our pipeline.

The strongest same-architecture competitor is MetricGAN+ itself, which reports PESQ 3.15. Its learned discriminator imitates PESQ directly, so it is the most direct alternative way to aim at the metric. We use its generator and masking function unchanged and replace only the discriminator signal with our evolved loss. Our total budget, search plus final training, is capped at a small multiple of MetricGAN+'s training compute in GPU-hours. Beating it means a mean PESQ above our re-run of MetricGAN+ over three seeds, with no STOI loss.

The strongest overall number in the file is CMGAN, which reports PESQ 3.41 with a Conformer generator. We transplant the evolved loss into CMGAN in place of its metric discriminator. Matching 3.41 with a fixed loss counts as success. The file reports no STOI values for these systems on VoiceBank-DEMAND, so all STOI numbers come from our re-runs.

For multiple metrics, the baseline is MetricGAN-OKD's multi-generator distillation. No numeric PESQ or STOI result was found on Lacuna for it. Beating it means a single generator matching both of its metrics.

Three search-method baselines share our exact query and GPU budget:

  • LossAgent-style weight tuning over a fixed base set.
  • CSE-Autoloss-style genetic programming over primitives, using our probe fitness. This isolates the effect of LLM operators from the effect of the cheap fitness.
  • REINFORCE in the style of Search-Softmax over a parameterized loss family.

The key ablations remove one ingredient at a time:

  • Hiding the metric code.
  • Replacing probe fitness with full-training fitness at equal compute, which means far fewer candidates.
  • Removing crossover.
  • The one-shot LLM loss with no evolution.

For imbalanced classification, we use KDD Cup 2008. There the adaptive-surrogate projection method of Optimizing Black-box Metrics with Adaptive Surrogates reports PRBEP 0.480 against 0.441 for the previous surrogate. Exceeding 0.480 with the same linear model class counts as beating it.

Why not just…?

Why not just use a learned metric surrogate like MetricGAN+? It is the real threat. On single-metric PESQ it may well tie us. The documented weaknesses are drift as the generator changes and conflicting gradients across multiple surrogates, the latter shown by MetricGAN-OKD. An evolved loss is fixed code and cannot drift. The joint PESQ+STOI experiment and the out-of-domain LibriSpeech noise test settle this question. If MetricGAN+ wins on both, our contribution shrinks to interpretability, and we will report it that way.

Why not just ask an LLM once to write a differentiable version of the metric? If that works, evolution is unnecessary. It is therefore a required ablation, scored with the same probe and the same final training. We expect one-shot relaxations to be brittle, because the hard parts of PESQ, such as time alignment and asymmetric disturbance, are exactly where a naive smoothing loses signal. That expectation is a prediction, not established evidence. If the one-shot gap is small, the proposal should be simplified to "metric-reading relaxation plus probe filtering."

Why not just give LossAgent a bigger library of base losses? The LossAgent authors note that its weight updates become erratic as the number of loss terms grows. More importantly, a weighted sum cannot express terms that are not in the library. The budget-matched LossAgent baseline, given a library that includes every seed loss we use, tests whether open-ended code adds anything beyond reweighting.

Why not just run Bayesian optimization or REINFORCE over a parameterized loss? These methods work in a fixed low-dimensional family, as MOEW and Search-Softmax do. They are strong when the right family is known in advance. The budget-matched REINFORCE baseline shows whether open-ended structure matters for metrics as complicated as PESQ.

Decisive Checks

Four results decide the proposal:

  • Probe validity. Across about 200 candidates, the Spearman correlation between one-step fitness and the PESQ reached after full training must be clearly positive. If it is near zero, the cheap fitness fails and the method fails with it.
  • LLM operators versus primitives. At equal budget, LLM operators must beat primitive-based genetic programming using the same probe fitness.
  • Single generator, two metrics. The single-generator PESQ+STOI loss must match multi-generator distillation on both metrics.
  • Value of reading the metric. Hiding the metric code must measurably hurt. If it does not, "reading the metric" is decoration, and the claim reduces to code-level loss evolution with a cheap fitness.