Differentiable Neural Surrogates for Non Differentiable Metrics

25 concepts20 papers3 proposals

Deep learning relies heavily on gradient descent, which requires every operation in a model's training loop to be differentiable. However, the metrics we actually use to evaluate these models—such as VMAF for video quality, PESQ for speech enhancement, or Panoptic Quality for image segmentation—are often discrete, rule-based, or algorithmic. Because these metrics lack analytical derivatives, models cannot optimize for them directly.

Instead, practitioners traditionally resort to mathematically convenient "surrogate" losses like Mean Squared Error (MSE) or cross-entropy. As highlighted by Huang et al., this creates a loss-metric mismatch: a model can successfully minimize its training loss while plateauing or even degrading on the actual evaluation metric of interest.

To resolve this mismatch, a growing body of research focuses on creating differentiable metric surrogates—smooth approximations of target metrics that allow gradients to flow directly from the evaluation criteria back into the primary model.


Bridging the Gap: Analytical Relaxation vs. Learned Proxies

There are two primary paradigms for constructing a differentiable surrogate: analytical relaxation and learned neural proxies.

Analytical Relaxation

In some cases, a discrete metric can be mathematically smoothed into a continuous approximation by replacing discrete operations (like thresholds or counts) with continuous functions. For instance, Kolkin et al. substituted discrete norms with continuous $L_2$ norms to approximate the spatially sensitive Weighted F-measure ($F_\beta^w$). Similarly, Abbas et al. developed a smooth surrogate for the Panoptic Quality metric to enable end-to-end training through a combinatorial solver, while Wang et al. relaxed discrete z-score detection metrics into differentiable forms to optimize large language model watermarking.

Learned Neural Proxies

When a metric is too complex, proprietary, or algorithmic to relax analytically (such as human perceptual metrics), researchers train a secondary neural network to act as the surrogate. Because neural networks are universal function approximators and are inherently differentiable, they can learn to mimic the behavior of any black-box evaluation metric.

[Primary Model] ---> Output (ŷ) ---> [Proxy Network] ---> Predicted Metric Score
       ^                                                          |
       |=================== Backpropagated Gradients =============|

During training, the primary model (e.g., a generator or enhancement network) produces an output. This output is passed into the trained proxy network, which predicts the metric score. Because the proxy network is differentiable, gradients can be calculated from the predicted score, backpropagated through the proxy's weights, and passed directly into the primary model. For example, Fu et al. introduced Quality-Net to approximate the PESQ metric, creating a learned loss function that allows speech enhancement models to directly optimize for human perceptual quality.


The Moving Target: Alternating Training Loops

A major challenge of using a learned neural proxy is distribution shift. If a proxy network is trained only once on a static dataset, the primary model will quickly learn to exploit the proxy's blind spots, generating unrealistic outputs that receive high predicted scores but low actual metric scores.

To prevent this, researchers employ alternating training frameworks similar to Generative Adversarial Networks (GANs). In this setup, the primary model and the proxy network are updated sequentially:

  1. Update the Proxy: The proxy network is trained to minimize the error between its predictions and the true, non-differentiable metric scores using the current outputs of the primary model.
  2. Update the Primary Model: The primary model is trained to maximize the score predicted by the proxy network, using the proxy's gradients to guide its updates.

Chen et al. utilized this alternating scheme in ProxIQA to approximate the VMAF metric for image compression, achieving significant bitrate reductions while maintaining perceived visual quality. Similarly, Xu et al. implemented an epoch-level alternating loop with PESQNet to optimize speech enhancement models.

In the context of super-resolution, Zhang et al. and Zhang et al. introduced a Siamese "Ranker" network within a GAN framework to mimic non-differentiable perceptual rankings, forcing the generator to optimize for complex visual traits. This adversarial surrogate approach is also central to the MetricGAN lineage, including work by Fu et al. (MetricGAN+) and Li et al. (iMetricGAN), where a discriminator learns to act as a surrogate for objective metrics like speech intelligibility.


Expanding Beyond Perceptual Quality

While initially prominent in image and speech processing, the use of differentiable surrogates spans multiple domains:

  • Unsupervised and Non-Intrusive Enhancement: Traditional quality metrics often require a clean reference signal. However, Xu et al. demonstrated that a non-intrusive neural proxy can evaluate speech quality without seeing clean audio. This concept is extended in MetricGAN-U, which leverages non-intrusive metrics to train models entirely on real-world, "in-the-wild" audio recordings.
  • Sequential Decision Making: Chen et al. integrated a value network as a differentiable proxy within a diffusion probabilistic model, optimizing speech enhancement as a sequential decision process via reinforcement learning.
  • Task-Specific Business Logic: Beyond perception, Chen et al. introduced TOPNet to approximate complex, non-differentiable financial evaluation criteria, aligning model optimization directly with real-world profit goals.
  • Adversarial Vulnerability Assessment: Burnyshev et al. utilized metric-approximating networks to calculate gradients for attacking Neural Machine Translation (NMT) models, demonstrating how surrogates can expose or test model robustness against non-differentiable text evaluation criteria.
  • Clustering: While fields like speech and vision have widely adopted these techniques, Beer notes that the potential for using pointwise clustering metrics as differentiable loss functions within deep learning models remains largely unexplored, highlighting an open frontier for unsupervised learning.

The Perils of Optimization: Goodhart's Law

While differentiable surrogates unlock powerful optimization capabilities, they are highly susceptible to Goodhart's Law: when a measure becomes a target, it ceases to be a good measure.

Because the surrogate is an approximation, the primary model can find mathematical loopholes—often called adversarial shortcuts—that maximize the metric score without improving the actual quality. For example, de Oliveira et al. demonstrated that optimizing speech enhancement models solely against a differentiable PESQ surrogate can lead to high objective scores driven by artificial artifacts (such as the "click trick"), while the actual audio sounds worse to human listeners.

This behavior underscores a critical lesson: a differentiable surrogate is only as good as its alignment with reality. To prevent models from "cheating" the surrogate, researchers must rely on multi-metric evaluation, regularized training, and continuous validation against true human or algorithmic baselines.

Key Foundational Work

MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement (2019) — contributed 2 concept elements to this direction.
ProxIQA: A Proxy Approach to Perceptual Optimization of Learned Image Compression (2019) — contributed 1 concept elements to this direction.
RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolution (2019) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_differentiable_metric_approximation_diff_fbb94122,
  title = {differentiable metric approximation (differentiable metric surrogacy)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/differentiable-metric-approximation-differentiable-metric-surrogacy/rdr_fbb94122c37b46b4b55c8bd3e1505eb2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_differentiable_metric_approximation_diff_fbb94122_v1,
  title = {differentiable metric approximation (differentiable metric surrogacy)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/differentiable-metric-approximation-differentiable-metric-surrogacy/rdr_fbb94122c37b46b4b55c8bd3e1505eb2/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake