Direct Optimization of Non-Differentiable Metrics

76 concepts64 papers3 proposals

Deep learning thrives on the elegance of the chain rule. To train a neural network, we typically require a smooth, continuous loss landscape where every tiny adjustment to a model's weights yields a predictable, gradient-defined change in performance. This requirement is why standard frameworks rely heavily on mathematically well-behaved objectives like Mean Squared Error (MSE) or Cross-Entropy.

However, a fundamental mismatch exists between these mathematically convenient loss functions and what humans actually care about. In speech processing, we evaluate models using complex algorithmic benchmarks like Perceptual Evaluation of Speech Quality (PESQ) or Short-Time Objective Intelligibility (STOI). In image generation, we prioritize structural similarity (SSIM) or human visual preference rankings. In broader applications, we optimize for discrete, non-decomposable metrics like F1-score, sequence alignment, or fairness constraints.

As noted by Xue et al. and Fu et al., these real-world evaluation metrics are frequently non-differentiable, non-convex, or step-like algorithmic functions. If you try to backpropagate through them directly, the gradients are either zero everywhere or completely non-existent. To overcome this limitation, researchers have developed three primary strategies to bridge the gap between gradient-based optimization and non-differentiable evaluation metrics.

Differentiable Neural Surrogates

If a metric cannot natively provide gradients, one powerful solution is to train a separate neural network to act as a proxy for that metric. Because neural networks are inherently differentiable, this "surrogate" network can ingest the model's outputs, predict what the non-differentiable metric score would be, and pass smooth, meaningful gradients back to the primary model.

[Generator/Model] ---> [Generated Output] ---> [Surrogate Network] ---> [Predicted Metric Score]
       ^                                                                        |
       |__________________________________ Backpropagated Gradients ____________|

This approach is highly effective in speech enhancement. To optimize networks for the non-differentiable PESQ metric, the authors of Learning With Learned Loss Function: Speech Enhancement With Quality-Net to Improve Perceptual Evaluation of Speech Quality introduced Quality-Net, a neural network trained specifically to approximate PESQ scores. By using Quality-Net as a differentiable loss function, the underlying enhancement model receives the granular feedback it needs to improve human-perceived audio quality.

This concept was subsequently expanded into an adversarial framework by Fu et al. with MetricGAN+. Here, a generator learns to enhance speech while a surrogate discriminator is trained to predict the non-differentiable metric. This allows the generator to directly maximize the discriminator's scores.

A similar strategy has solved long-standing optimization hurdles in computer vision. For instance, Zhang et al. and Liu developed RankSRGAN for image super-resolution. They trained a differentiable "Ranker" network to mimic non-differentiable perceptual metrics, successfully guiding Generative Adversarial Networks (GANs) to produce images tailored to human visual preferences.

To make these surrogates even more versatile, Xu demonstrated that a non-intrusive surrogate network can optimize speech models even when clean reference signals are entirely missing from the training data, opening the door to self-supervised perceptual optimization.

Differentiable Relaxations and Smooth Proxies

Instead of training a secondary neural network to learn a metric, another strategy mathematically alters or "relaxes" the metric itself. The goal is to strip away the discontinuous or step-like operations of an algorithmic metric, replacing them with smooth, continuous approximations that preserve the metric's perceptual essence.

A clear example of this mathematical smoothing is found in the work of Xue et al., who introduced Perceptual-fidelity Aware Mean Squared Error (PAMSE). While standard MSE fails to capture human visual perception, highly accurate metrics like SSIM are non-linear and non-convex. The authors resolved this by applying a simple Gaussian smoothing filter to the images before calculating standard Euclidean distances. This minor, fully differentiable modification yields a loss function that behaves as smoothly as MSE during training but achieves the perceptual accuracy of SSIM.

In speech synthesis and enhancement, many expert-defined acoustic parameters—such as pitch, jitter, and shimmer—are calculated using non-differentiable, discrete algorithms. To inject this domain knowledge directly into gradient descent pipelines, Yang designed PAAPLoss, and Zeng et al. developed TAPLoss. Both frameworks utilize differentiable neural estimators to track fine-grained temporal acoustic parameters, penalizing the model directly when its outputs violate acoustic realism.

Similarly, in computer vision, Shechtman formulated a differentiable approximation of the Weighted F-measure ($F_\beta^w$). This allowed deep networks to optimize for complex spatial contexts—such as penalizing internal holes in object segmentation more harshly than fuzzy boundaries—without relying on slow, non-differentiable post-processing techniques like Conditional Random Fields.

Bypassing Gradients via Reinforcement Learning and Black-Box Optimization

When an evaluation metric is a complete "black box"—meaning its mathematical formulation is entirely unknown, non-decomposable, or relies on discrete human feedback—neither neural surrogates nor analytical relaxations may be feasible. In these cases, researchers bypass gradient computation entirely, treating the metric as an environment and the deep network as an agent.

As surveyed by Uc-Cetina et al., Reinforcement Learning (RL) serves as a vital tool for optimizing non-differentiable text metrics like language fluency, semantic similarity, or user satisfaction. Because text generation involves a sequence of discrete token selections, standard backpropagation breaks down. RL circumvents this by treating the discrete metric score as a scalar reward, optimizing the model's generation policy over time.

This paradigm is equally powerful in audio processing. Chen et al. developed the MOSE framework, which frames the iterative denoising steps of a speech diffusion model as a sequential decision process. By applying RL, they successfully optimized the diffusion model against the non-differentiable PESQ metric without needing to approximate the metric itself.

Alternatively, black-box metrics can be optimized by changing how data is fed to the model rather than changing the model's internal gradient calculations. The authors of Optimizing Black-box Metrics with Adaptive Surrogates demonstrated that complex, non-decomposable global constraints (like fairness or specific F1-score targets) can be optimized by mapping them to a low-dimensional combination of simple, standard surrogate losses.

Taking this concept further, Zhao introduced the Metric-Optimized Example Weights (MOEW) framework. Instead of altering the loss function, MOEW uses Bayesian optimization to dynamically learn how much weight to assign to individual training examples. By strategically emphasizing or de-emphasizing specific data points based on validation performance, the framework maximizes arbitrary, black-box test metrics while leaving the underlying gradient descent mechanics completely untouched.

Key Foundational Work

Cepstral analysis based on the glimpse proportion measure for improving the intelligibility of HMM-based synthetic speech in noise (2012) — contributed 1 concept elements to this direction.
Perceptual Fidelity Aware Mean Squared Error (2013) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 67 papers
  2. Version 1 · 2026-10-04 · 96 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_non_differentiable_objective_functions_n_c0d430cf,
  title = {non-differentiable objective functions (non-differentiable metric optimization)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-differentiable-objective-functions-non-differentiable-metric-optimization/rdr_c0d430cf09d94fe19c6038771779a75f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 2, frozen text):

@misc{lacuna_non_differentiable_objective_functions_n_c0d430cf_v2,
  title = {non-differentiable objective functions (non-differentiable metric optimization)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-differentiable-objective-functions-non-differentiable-metric-optimization/rdr_c0d430cf09d94fe19c6038771779a75f/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake