Policy Refinement via Control Interface Functions

29 concepts18 papers1 proposal

When designing controllers for real-world cyber-physical systems—such as autonomous vehicles, robotics, or smart building grids—we are immediately confronted by the curse of dimensionality. Real systems operate in continuous state spaces, experience stochastic noise, and exhibit complex nonlinear dynamics. Synthesizing control policies directly on these high-dimensional models is often computationally intractable.

To bypass this complexity, a powerful paradigm has emerged: Abstraction-Based Control. Instead of solving the control problem on the complex physical system directly, we construct a simplified, discrete, or lower-dimensional "abstract" model. We then synthesize a policy in this simpler domain where the mathematics is tractable, and finally translate—or refine—that policy back to the original system.

The core challenge of this paradigm is ensuring that the safety and performance guarantees proven on the simplified model actually hold when the refined policy is deployed in the real world. This is achieved through a mathematical bridge known as a control interface function.

The Abstraction-Refinement Pipeline

The entire process can be viewed as a two-player coordination problem between the abstract model and the concrete (physical) system. As noted by Nathalie Cauchi in Benchmarks for cyber-physical systems: A modular model library for building automation systems, using abstractions like Markov Decision Processes (MDPs) makes policy synthesis computationally manageable by capturing only the essential dynamics of a system.

The pipeline operates in three distinct phases:

  1. Abstraction: The continuous or high-dimensional state space is partitioned into a finite set of symbolic states. Rupak Majumdar et al. in Counterexample-Guided Control demonstrated that techniques like Counterexample-Guided Abstraction Refinement (CEGAR) can automatically discover "just right" abstractions—coarse enough for tractability, yet detailed enough to preserve control objectives.
  2. Synthesis: A high-level policy is computed on the abstract model. For instance, Kaushik Mallik et al. in Symbolic control for stochastic systems via finite parity games show how complex temporal logic specifications can be solved by transforming the abstract model into a finite parity game, yielding a winning strategy with a guaranteed lower bound on success.
  3. Refinement: The abstract strategy is mapped back to the physical system. Matthias Rungger et al. in A Notion of Robustness for Cyber-Physical Systems formalize this as a systematic framework that takes a robust controller synthesized for a symbolic model and translates it back into concrete execution commands.

The Mechanism of Control Interface Functions

An abstract policy tells you what symbolic action to take given a symbolic state (e.g., "if in region $A$, move to region $B$"). However, a physical actuator requires precise, continuous inputs (e.g., a specific voltage or force vector).

The control interface function is the mathematical mapping that resolves this mismatch. It takes two inputs—the current state of the concrete system and the desired action from the abstract policy—and outputs the exact control input for the physical system.

[Abstract State] -------> [Abstract Policy] -------> [Abstract Action]
                                                           |
                                                           v
[Concrete State] ----------------------------------> [Interface Function]
                                                           |
                                                           v
                                                    [Concrete Input]

As explained by Abolfazl Lavaei et al. in From Dissipativity Theory to Compositional Construction of Finite Markov Decision Processes, this interface function acts as a local controller. Its job is to minimize the deviation between the physical system's actual trajectory and the idealized trajectory of the abstract model.

By parameterizing the concrete input based on both the abstract decision and the real-time state error, the interface function forces the physical system to "track" the abstract system. Sofie Haesaert et al. in Robust Dynamic Programming for Temporal Logic Control of Stochastic Systems emphasize that this mechanism ensures the physical system's trajectory remains within a strictly bounded distance of the abstract trajectory.

This concept is highly versatile. For example, Alireza Nadali et al. in Transfer Learning for Control Systems via Neural Simulation Relations demonstrate that interface functions can be parameterized by neural networks to synchronize behaviors between a simulator (source) and a real-world robot (target) within proven error margins.

Why It Works: Quantifying the Guarantee Gap

The true power of this framework lies in its ability to provide correct-by-construction guarantees. We do not merely hope the physical system behaves like the abstraction; we mathematically bound how far it can stray.

This is achieved by establishing formal relations—such as approximate similarity or simulation relations—between the two models. When an interface function is successfully constructed, it proves the existence of a simulation relation, which provides a quantifiable upper bound on the approximation error.

Preserving Probabilistic Thresholds

In stochastic systems, such as general Markov Decision Processes (gMDPs), the guarantees are probabilistic. Sadegh Soudjani et al. in Verification of general Markov decision processes by approximate similarity relations and policy refinement demonstrate that if a safety property is verified to hold with a certain probability on the abstraction, the policy refinement process guarantees that the probability of safety on the original continuous system stays above a calculated threshold. This threshold is a direct function of the approximation error and the time horizon.

Sofie Haesaert et al. in Verification of General Markov Decision Processes by Approximate Similarity Relations and Policy Refinement show that these approximate similarity relations allow designers to systematically trade off model complexity against the tightness of the performance bounds.

Handling Uncertainty and Scale

The framework scales effectively to complex settings by decomposing the error analysis:

Ultimately, policy refinement via interface functions changes the control design problem. As summarized by Thom Badings et al. in Probabilistic Alternating Simulations for Policy Synthesis in Uncertain Stochastic Dynamical Systems, it guarantees that the actual probability of satisfying safety requirements in a real-world deployment is mathematically bounded by—and often at least as high as—the success rate predicted by the simplified abstract model.

Key Foundational Work

Symbolic Models for Nonlinear Control Systems: Alternating Approximate Bisimulations (2009) — contributed 1 concept elements to this direction.
Counterexample-Guided Control (2003) — contributed 1 concept elements to this direction.
Hierarchical trajectory generation for a class of nonlinear systems (2003) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_controller_refinement_3b997b11,
  title = {controller refinement},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/controller-refinement/rdr_3b997b11276848c9958517f12053d9c1},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_controller_refinement_3b997b11_v1,
  title = {controller refinement},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/controller-refinement/rdr_3b997b11276848c9958517f12053d9c1/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake