Multi-objective drift-aware MLOps agent for model retraining

Kumuda Valli, Anli JiarXiv 2026 · DOI · arXiv

In the world of production machine learning, "set it and forget it" is a recipe for disaster. Data distributions shift, models degrade, and performance eventually falls off a cliff—a phenomenon known as concept drift. Traditionally, MLOps teams handle this with blunt instruments: retraining on a fixed schedule (e.g., every Monday) or setting a static drift threshold.

However, a new paper from researchers at California State University, Fullerton, argues that these heuristics are fundamentally flawed because they ignore the operational cost of retraining. They introduce DARE (Drift-Aware Retraining Engine), a framework that reframes retraining not as a simple trigger, but as a sequential decision problem.

TL;DR

DARE treats model retraining as a cost-aware sequential decision problem using contextual bandits rather than static thresholds. By balancing predictive performance against operational overhead, DARE achieves higher F1 scores while reducing cumulative retraining runtime by over 80% compared to standard heuristic baselines.

What Problem Does This Paper Address?

Most retraining strategies are "brittle." A drift threshold that works well during stable periods might over-trigger during a volatile week, wasting expensive compute resources and engineering effort. Conversely, fixed schedules might miss sudden, critical shifts in data.

Furthermore, real-world MLOps faces the "delayed label" problem: you often don't know if your model was right until days or weeks later. Most existing systems don't explicitly account for this delay or the actual wall-clock cost of running a training pipeline. The authors identify a gap: we need a way to decide when to retrain that considers the trade-off between model utility and pipeline overhead.

Methods

The authors propose an event-driven architecture called DARE. Its core philosophy is the strict separation of monitoring, policy decisions, and execution.

Fig. 1. DARE event-driven pipeline. Dashed arrows indicate validation-failure routing to the DLQ.

Key technical components include:

  • Sequential Decision Framing: Instead of asking "has drift crossed a threshold?", the system asks "given the current state, should I defer or fine-tune?"
  • LinUCB Policy: DARE employs a Linear Upper Confidence Bound (LinUCB) contextual bandit. This agent learns a policy that maps system states (drift features, performance metrics) to actions.
  • Multi-objective Reward: The agent is trained using a reward function that rewards performance gains ($\Delta F1$) but penalizes measured runtime cost and the act of intervening itself.
  • Deterministic Replay: To ensure fair comparisons, the framework uses an offline runtime that can "replay" historical data streams deterministically, allowing different policies to be tested under identical conditions.

Datasets

The study evaluates DARE on two temporally ordered benchmarks:

  1. Airline 120k (Primary): A binary classification task predicting flight delays (>15 mins). It is notably class-imbalanced (only ~17.5% delays), making it a challenging real-world proxy.
  2. Elec2: A benchmark derived from electricity market signals, used primarily for secondary calibration and robustness testing.

Key Insights and Ideas

  • Retraining is a Policy, Not a Trigger: The biggest "aha" moment is treating retraining as a learned behavior. By framing it as a bandit problem, the model learns to be "conservative"—only updating when the expected performance gain outweighs the cost.
  • Timing vs. Frequency: To prove DARE wasn't just winning by "updating less," the authors created an "action-rate-matched" baseline. Even when a fixed schedule was tuned to update at the exact same frequency as DARE, DARE still performed better. This proves that when you update is just as important as how often.
  • Metric Alignment is Everything: The researchers found that optimizing for simple Accuracy in imbalanced datasets led the agent to "over-intervene" without improving the metrics that actually matter. Switching the reward to $\Delta F1$ immediately restored efficient, conservative behavior.
  • Robustness to Delay: Surprisingly, the agent remained stable even with label delays of up to 10 batches. In fact, under high delay, the agent naturally became more conservative, which actually led to a slight increase in F1 on the Airline task.

Key Results

The quantitative results (summarized in Table I) show a clear victory for the learned policy approach.

Comparison of different retraining policies. LinUCB achieves the highest F1 with significantly lower runtime than heuristic baselines.
  • Efficiency: Compared to a standard fixed-interval schedule, DARE reduced cumulative update runtime by 88.3%.
  • Performance: Despite the massive reduction in compute, DARE achieved a higher mean F1 (0.2065) than all other adaptive policies.
  • Pareto Dominance: As shown in the trade-off plot (Fig 2), DARE occupies the "ideal" top-left quadrant—high performance, low cost.
The Pareto trade-off shows LinUCB (green) dominating other strategies by providing better F1 scores at a fraction of the operational cost.

Limitations and Future Work

While promising, the authors note several areas for expansion:

  • Hardware Specificity: Runtime was measured on a single hardware configuration; absolute times might vary in different cloud environments.
  • Action Space: The current study focused on a binary choice (defer vs. fine-tune). Future work could include "full retrain" or "hyperparameter search" as more expensive, high-reward actions.
  • Fairness: While DARE logs fairness metrics, they weren't used as primary optimization constraints in this study. The authors aim to couple fairness signals directly to action selection in the future.
  • Generalization: The current evaluation uses tabular data; testing DARE on high-dimensional image or text streams remains an open question.

Summary updated

Abstract

Deployed ML models face non-stationary data and delayed labels, yet most retraining strategies rely on fixed schedules or brittle drift thresholds that ignore operational cost. In this setting, retraining is a sequential decision problem rather than a one-time threshold choice. We present DARE (Drift-Aware Retraining Engine), an offline, event-driven retraining framework that separates ingestion, monitoring, policy decisions, training, and logging through explicit schemas and deterministic replay. We compare DARE against practical heuristic retraining baselines, including fixed-interval, threshold-persistence, and action-rate-matched fixed scheduling policies. On Airline, the locked LinUCB policy achieves a higher mean F1 while substantially reducing intervention frequency relative to heuristic and action-rate-matched baselines. Additional runtime-calibrated evaluation shows that these gains persist when measured using the cumulative update wall-clock time. These results suggest that conservative sequential retraining policies can improve performance while reducing unnecessary operational overhead under delayed feedback.
BibTeX
@misc{valli2026multi,
  author = {Kumuda Valli and Anli Ji},
  title = {{Multi-objective drift-aware MLOps agent for model retraining}},
  year = {2026},
  eprint = {2611.10313},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.10313},
}
Figure