Iterative Model Refinement from Closed Loop Data

24 concepts19 papers1 proposal

When deploying autonomous systems in the real world—whether racing a drone through a tight gate sequence or maneuvering a robotic arm at high speeds—we inevitably run into the limits of physics-based models. A static, analytical model of a vehicle cannot perfectly capture complex, highly dynamic aerodynamic disturbances or non-linear tire friction.

Traditionally, control engineers handle this "model mismatch" by either slowing the system down to stay within conservative safety margins or spending weeks hand-tuning controllers. Iterative model refinement offers a more elegant alternative: instead of trying to build a perfect global model upfront, the system executes a task, observes where its current model fails, and uses that closed-loop data to progressively refine its internal models and control inputs for the next run.

By shifting the burden from perfect prior modeling to continuous, data-driven adaptation, systems can safely push themselves to the absolute edge of their physical capabilities.

Isolating and Learning the Residual Error

The core intuition behind iterative refinement is that we do not need to learn the entire dynamics of the universe from scratch; we only need to learn how our simplified nominal model mismatches reality along the specific path we want to take.

Rather than discarding a physics-based model, hybrid frameworks treat it as a reliable baseline and layer a data-driven model on top to capture the residual errors. For instance, Balta et al. combined Iterative Learning Control (ILC) with Gaussian Process Regression (GPR) to explicitly learn the hardware-to-model mismatch from run-to-run measurements, reducing tracking errors by over 70%. Similarly, in high-speed autonomous racing, Magno et al. demonstrated an on-track system identification method that pairs a traditional tire model with a neural network. By iteratively updating the nominal parameters and retraining the network on the remaining residual errors, the vehicle adapts its dynamics model to a new track in under a minute.

This strategy allows the system to remain robustly anchored by known physics while using empirical data to absorb complex, unmodeled disturbances.

Continuous vs. Episodic Adaptation

Depending on how data is gathered, iterative refinement operates in one of two modes: episodic (run-to-run) or continuous (online).

In episodic learning, updates happen between distinct trials or laps. The system completes an entire trajectory, analyzes the performance history, and updates its strategy before starting the next attempt. Dorobantu et al. applied this to uncertain robotic systems by using episodic data to update uncertainty estimates within a Control Lyapunov Function framework, gradually enhancing stability guarantees.

In contrast, continuous adaptation refines the model while the feedback controller is actively running. Lale et al. proved that continuously updating system estimates using active closed-loop data significantly outperforms traditional "explore-and-commit" strategies, which rely on a fixed, isolated exploration phase. To ensure this continuous updating remains stable, Balta et al. integrated an online gradient descent mechanism to minimize model mismatch dynamically, proving that such a mechanism achieves sublinear regret—meaning the controller rapidly "catches up" to the mathematically optimal control sequence over time.

Whether updating between laps or between time steps, the underlying mechanism is identical: leveraging the correlation in operational data to squeeze out uncertainty.

Self-Improvement from Arbitrary Baselines

A remarkable property of iterative refinement is its ability to optimize performance without requiring a human-provided reference trajectory or a perfect initial guess.

Because the refinement loop evaluates the success of previous iterations, it can guide a system toward local optimality autonomously. Jafarzadeh et al. showed that data-driven predictive control frameworks can autonomously discover and refine sub-optimal paths even when parts of the environment or cost functions are entirely unknown.

This capacity for self-improvement is vividly illustrated in autonomous drone racing. As surveyed by Betz et al., iterative learning mechanisms allow vehicles to progressively optimize control inputs by updating track and vehicle models lap after lap. Building on this concept, Zhao et al. introduced an iterative learning Model Predictive Control (MPC) framework that can be initialized with any successful baseline trajectory. By using data from previous iterations to update the cost-to-go function and safe state boundaries, the system consistently drives down lap times. Remarkably, even when initialized with a highly optimized, state-of-the-art, hand-tuned controller, the iterative loop still extracts an additional 6% performance gain.

[Initial Baseline Trajectory] 
           │
           ▼
┌─────────────────────────────────────┐
│  Execute Maneuver & Collect Data    │◄────────────────┐
└─────────────────────────────────────┘                 │
           │                                            │ Iterative
           ▼                                            │ Refinement
┌─────────────────────────────────────┐                 │ Loop
│  Isolate Mismatch & Update Model    │                 │
└─────────────────────────────────────┘                 │
           │                                            │
           ▼                                            │
┌─────────────────────────────────────┐                 │
│ Optimize Cost-to-Go & Safe Sets     │─────────────────┘
└─────────────────────────────────────┘
           │
           ▼
[Locally Optimal, High-Speed Execution]

The Interdependence of Control and Identification

Iterative model refinement elegantly resolves a classic control paradox: dual control. To learn an accurate model of a system, you need highly exciting, varied data (exploration). But to keep the system safe and performant, you must restrict its behavior (exploitation).

Iterative frameworks turn this tension into a cooperative loop:

  1. Synthesize a conservative controller based on the current model uncertainty.
  2. Run the system to collect high-quality, closed-loop operational data.
  3. Update the model and shrink the uncertainty bounds.
  4. Synthesize a more aggressive, high-performance controller.

This loop acknowledges that model accuracy and controller design are fundamentally coupled. For instance, Liu et al. utilized a prior system model to optimize "space-filling" input signals, ensuring that experimental data thoroughly covers the operating space to maximize model generalization. For safety-critical systems, Liu et al. unified formal controller synthesis with reachset-conformant identification, guaranteeing that the identified model's reachable states always contain all observed physical behaviors. This incremental tightening of the model set allows for precise safety verification without over-conservative performance degradation, a concept also advanced by Zhang et al. through data-driven non-convex reachability analysis.

This closed-loop operational cycle is highly versatile. It can be applied to discrete-time logical systems, where Gol et al. used an iterative loop to mine the causes of unwanted events and synthesize corrective controllers. It can even be applied to macro-scale mission planning: Morris et al. developed a closed-loop system for satellite and flight planning that identifies discrepancies between Earth science models and actual observations, automatically turning these "surprises" into optimized subsequent flight plans to target areas of high uncertainty. Even in software infrastructure, Becker et al. demonstrated that iteratively refining a runtime model based on data availability allows black-box machine learning services to adaptively allocate resources on unknown hardware.

Why It Works: Bypassing Higher-Order Errors

The empirical success of iterative learning control across these diverse domains raises a fundamental theoretical question: Why does explicitly updating models via iterative rollouts perform so much better than standard model-based control?

The answer lies in how errors propagate through a controller. In a purely static model-based setup, any initial modeling error compounds over the control horizon, forcing the controller to combat complex, higher-order error terms. Vemula et al. provided a rigorous theoretical foundation for this phenomenon. By deriving new perturbation bounds for the finite-horizon Riccati equation, the authors proved that iterative learning control effectively bypasses the higher-order error terms that cause traditional controllers to fail in the presence of large modeling errors.

By grounding the control sequence directly in the real-world data of previous rollouts, iterative refinement ensures that the system corrects its course based on actual experienced dynamics rather than compounding errors through an imperfect mathematical proxy. Early overviews of the field, such as those by Scaramuzza et al., recognized that this mechanism allows lightweight autonomous platforms to master aggressive maneuvers that far exceed the capabilities of static analytical models. Iterative model refinement provides a mathematically sound, highly practical blueprint for systems that learn to excel through practice.

Key Foundational Work

Special issue on micro-UAV perception and control (2012) — contributed 2 concept elements to this direction.
Learning to Improve Earth Observation Flight Planning (2008) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_iterative_model_refinement_e4965656,
  title = {iterative model refinement},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/iterative-model-refinement/rdr_e49656569cb1413e9f18a59fe6d51938},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_iterative_model_refinement_e4965656_v1,
  title = {iterative model refinement},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/iterative-model-refinement/rdr_e49656569cb1413e9f18a59fe6d51938/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake