Decision Focused Learning for Integrated Optimization

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2297 papers394 conceptsfrozen 2026-10-04

In traditional machine learning, we are taught to minimize statistical divergence. If we are predicting traffic patterns or energy prices, we aim for the lowest Mean Squared Error (MSE). However, in many industrial and scientific contexts, a prediction is not the final product—it is merely an input to a downstream optimization problem.

A delivery company doesn't just want to know the traffic; they want the shortest route. A power grid operator doesn't just want the weather forecast; they want a stable, low-cost generation schedule. This "Predict-then-Optimize" pipeline reveals a fundamental flaw in standard ML: statistical accuracy does not always equal decision quality.

The Mismatch Between Accuracy and Regret

The core problem, as identified by Christian Bessiere and Luc De Raedt in The Inductive Constraint Programming Loop, is that standard loss functions are "decision-blind."

Imagine an optimization problem where you must choose the cheapest path across a network. If your model makes a 10% error on the price of a road that was never going to be part of the optimal path anyway, the "regret"—the difference between your decision's cost and the true optimal cost—is zero. But if your model makes a tiny 1% error on a critical bottleneck, it might flip the entire decision, leading to a massive spike in real-world costs.

As Adam N. Elmachtoub and Paul Grigas argue in their work on Smart Predict-then-Optimize (SPO), MSE treats these two errors as equally important. Decision-Focused Learning (DFL) instead reshapes the loss landscape to prioritize accuracy where it actually changes the optimal solution.

Differentiating Through the Solver

To train a model this way, we need to pass the "regret" from the solver back to the neural network. This is mathematically difficult because optimization solvers (like those for linear programming or integer programming) are often non-differentiable "black boxes." Their outputs are discrete or piecewise constant; if you change a predicted parameter slightly, the optimal decision might stay exactly the same or jump suddenly to a new point.

Researchers have developed several clever ways to bridge this gap:

  1. Continuous Relaxations: Bryan Wilder and colleagues in Melding the Data-Decisions Pipeline propose differentiating through a continuous relaxation of the combinatorial problem. By smoothing the optimization landscape, they allow gradients to flow back to the predictive model.
  2. Interior Point Methods: Jayanta Mandi and Tias Guns demonstrate in Interior Point Solving for LP-based prediction+optimisation that the logarithmic barrier functions used in interior point solvers can serve as a natural, differentiable surrogate for the optimization task.
  3. Surrogate Losses: When the solver is too complex to differentiate, we can learn a "proxy" loss. Sanket Shah and Andrew Perrault introduced Locally Optimized Decision Losses (LODL), which uses the solver to generate data points and then trains a simpler convex function to mimic the solver's behavior during the ML training loop.

Robustness and Generalization

A recurring theme in recent work is that DFL is more resilient to model misspecification than two-stage pipelines. In Contextual Linear Optimization with Bandit Feedback, Nathan Kallus and Xiaojie Mao show that if your model class is "wrong" (e.g., you use a linear model for a non-linear reality), minimizing MSE will converge to a model that is mediocre everywhere. However, DFL will converge to a model that is accurate specifically in the regions that matter for the decision, leading to much lower regret.

This focus on the "utility" of information is further refined by Mehran Poursoltani and Erick Delage, who propose maximizing the coefficient of prescriptiveness—a metric that measures how much a specific piece of data actually improves the final decision.

Scaling to Real-World Systems

The shift from "accuracy-first" to "decision-first" is already transforming infrastructure. In power systems, Haipeng Zhang and Ran Li have shown that Decision-Oriented Learning can reduce operational costs by accounting for the physical constraints of the grid during the forecasting stage.

By treating the machine learning model and the optimizer as a single, unified architecture—as surveyed by Ferdinando Fioretto and Pascal Van Hentenryck in End-to-End Constrained Optimization Learning—we move away from models that are merely "right" about the data, and toward models that are "useful" for the world.

Papers in this version

Showing 60 of 297 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 264 papers
  2. Version 1 · 2026-10-04 · 297 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_predict_then_optimize_gap_51133ccd,
  title = {predict-then-optimize gap},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/predict-then-optimize-gap/rdr_51133ccd323d480eb134df1491648e90},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_decision_focused_learning_for_integrated_51133ccd_v1,
  title = {Decision Focused Learning for Integrated Optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/decision-focused-learning-for-integrated-optimization/rdr_51133ccd323d480eb134df1491648e90/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}