Large language models can pull structured facts out of clinical free text, such as problems, medications, and whether a finding is present or denied. They are made more reliable by asking them to check their own work. Self-Verification Improves Few-Shot Clinical Information Extraction runs a fixed sequence of steps: extract, look for omissions, find supporting evidence, prune. EHRCon chains eight prompted stages, including a self-correction step. These loops have no rule saying when a correction is actually an improvement. They can therefore re-add an item that a previous step pruned, swap one error for another, or spend the same large compute budget on easy and hard notes alike. The cost of that last problem is concrete. Information Extraction from Clinical Notes: Are We Ready to Switch to Large Language Models? reports 80.6 seconds per note for a 70-billion-parameter model.
We propose to borrow the central idea of a Lyapunov stability proof. In such a proof you exhibit a nonnegative "energy" score that must go down at every step, which guarantees the system settles. Here we define an energy over a note's current extraction that can be computed without gold labels. The model proposes edits, and we accept only edits that lower the energy. Two things follow. The loop provably terminates and never cycles. And the measured average energy drop per step gives each note its own stopping rule and compute budget. The open empirical question is whether "energy reached zero" actually tracks extraction correctness, and the experiments below are built around that risk.
Core Claim
The claim is that a hand-built, label-free energy function plus an accept-only-if-lower rule makes LLM self-correction for clinical extraction stable and adaptive. Stable means the loop never oscillates and terminates in bounded steps. Adaptive means it spends extra model calls only on notes that need them. We claim this matches or beats fixed-length verification pipelines at a fraction of their average cost.
The energy for a note with current extraction $x$ is
$$V(x) = g(x) + \lambda\, c(x) + \mu\, s(x),$$
where each term is a count checked by deterministic code rather than by the LLM:
- $g(x)$ counts extracted items whose claimed supporting span does not appear verbatim in the note.
- $c(x)$ counts mentions flagged by a cheap high-recall detector that no extracted item covers. For this detector we use a fine-tuned BERT tagger, the fast model class from the Kiwi study above.
- $s(x)$ counts schema or assertion violations. Examples are a child field filled while its parent is marked absent, in the style of the ontology graph used by mCODEGPT, or an item labeled "present" whose span sits inside a negation cue's scope.
- $\lambda$ and $\mu$ are nonnegative weights tuned on a development split.
Because $V$ is a nonnegative integer combination that the accept rule forces to decrease strictly, the loop halts in at most $V(x_0)/\delta$ accepted steps. Here $x_0$ is the first-pass extraction and $\delta$ is the smallest possible decrease.
Rejected proposals cost compute but do not change the state. For budgeting we therefore use the stochastic version of the argument, the Foster–Lyapunov drift condition. Suppose that, away from the "good" set $C = \{x : V(x) = 0\}$, the expected decrease per model call satisfies
$$\mathbb{E}[V(x_{k+1}) - V(x_k) \mid x_k] \le -\epsilon .$$
Then the expected number of calls $T$ to reach $C$ obeys $\mathbb{E}[T] \le V(x_0)/\epsilon$. This is the standard hitting-time consequence of negative drift, the same tool used for queue stability in Load Balancing via Random Local Search. We estimate $\epsilon$ per edit type on development notes. This turns the initial energy of a new note into a predicted budget, with a hard cap.
Classical Lyapunov analysis also has a notion for trajectories that stop improving without reaching the goal: under LaSalle's invariance principle they settle into the set where the energy stops decreasing. Our analogue is notes where no proposal lowers $V$ above zero. We treat these as a diagnostic class and route them to human review, rather than hiding them.
Relation to Prior Work
Two lines of work motivate the gap. The self-verification loop of Gero et al. and the eight-stage CheckEHR pipeline in EHRCon show that multi-step checking raises LLM extraction quality. EHRCon reports recall moving from about 10% to 61.06%. Neither defines a quantity that each stage must improve, so neither can rule out cycling or justify its step count. We keep their kinds of steps (omission search, grounding, pruning) as the proposal operators and add the acceptance test and drift-based budget they lack.
The Lyapunov literature supplies the method but also constrains it. Neural Lyapunov Control learns an energy function with a neural network and certifies it with an SMT solver, a tool that checks logical formulas over continuous variables. The authors note that this verification is intractable beyond small continuous state spaces. We deliberately avoid learning $V$. A discrete, hand-specified energy is trivially checkable, and that is what lets the certificate survive a token-level state space. Sheaf-Guarded Updates shares the spirit of guarding each update to an evolving agent state against a structural check. We specialize that idea to clinical extraction and add the drift-to-budget link.
Lyapunov work on clinical-adjacent dynamics does not touch text. Examples are the epileptic bistability analysis in Analytical Characterization of Epileptic Dynamics in a Bistable System and the recurrent-network Lyapunov spectra in Lyapunov spectra of chaotic recurrent neural networks. They are not competitors.
Research Program
We will build the loop around one open-weight instruction-tuned LLM and three proposal operators, each targeting one term of $V$:
- An add operator that tries to cover uncovered candidate mentions.
- A ground-or-drop operator for ungrounded items.
- A repair operator for schema and assertion violations.
At each step the operator targeting the largest weighted term proposes an edit. Code recomputes $V$ and keeps the edit only if $V$ falls.
The analysis has three parts. First, we measure the empirical drift $\epsilon$ per operator and test whether the predicted budget $V(x_0)/\epsilon$ matches realized call counts. Second, we characterize the stall set: which notes get stuck, and why. Third, and most important, we measure the rank correlation between the final $V$ and per-note F1 against gold annotations. A near-zero correlation would mean the energy optimizes the wrong thing.
The obvious way to game $V$ is to delete everything. The coverage term blocks this, since deleting a correct item uncovers a detector mention and raises $c$. Weakly grounded but wrong items, such as a verbatim span with the wrong assertion, are the residual failure. We will quantify them by error type.
Baselines
The naive reference is a single-pass LLM extraction with the same prompt and model and no verification. This shows what any checking buys.
The detector-only reference is the fine-tuned BERT tagger that also supplies $c(x)$. It is trained and evaluated on the same splits. On 2010 i2b2 cross-institution evaluation, the Kiwi study reports BERT at F1 0.798 and LLaMA-3-70B at 0.872. For in-distribution i2b2, Joint Entity Extraction and Assertion Detection for Clinical Text reports entity F1 0.855 and negation F1 0.905 with a joint neural tagger. These are the strongest reported i2b2 numbers in our file and the bar for the trained-model regime.
The strongest LLM competitor is the four-step self-verification pipeline of Gero et al. We reimplement it with the same base model and prompt templates and give it the same maximum number of model calls. No reported result found on Lacuna for self-verification on 2010 i2b2, so we will report our reimplementation. On EHRCon, the competitor is CheckEHR at its reported 61.06% few-shot recall. We run our loop in place of CheckEHR's self-correction stage with everything else held fixed.
The key ablation is the identical loop without the acceptance rule, where every proposed edit is applied. It isolates the contribution of the descent test. A second ablation is a fixed-iteration loop capped at the same average call count our adaptive stopping uses, which isolates the value of the drift-based budget.
Beating the self-verification pipeline means equal or higher F1 at lower average model calls per note, or higher F1 at equal calls, with bootstrap confidence intervals over notes.
Why not just…?
Why not just run the fixed self-verification pipeline once? It is the most likely real threat. If its single omission-then-prune pass already lands near the accuracy ceiling, iterating adds little. Our same-budget comparison settles this. The claimed advantage is not higher peak F1 but equal F1 at lower average cost, because easy notes (small $V(x_0)$) stop after one pass. If adaptive stopping saves no calls, or loses F1 at matched calls, the proposal fails on its own terms.
Why not just cap iterations at k and skip the Lyapunov story? A cap guarantees termination but not improvement. An uncapped or capped loop without an acceptance test can still re-add pruned items or trade errors. The accept-all ablation measures exactly this, by counting items that flip in and out and tracking F1 across iterations. If that loop is empirically monotone anyway, the acceptance rule is unnecessary and we will say so. What would remain is the drift-based per-note budget, which a fixed cap cannot provide.
Why not just take the BERT detector's high-recall output, union it with one LLM pass, and deterministically drop ungrounded items? This one-shot filter-and-union uses the same ingredients as $V$ with no loop. It could plausibly match us, so we run it as a baseline. If it does match, the loop's value reduces to repairing schema and assertion violations. We will report results broken down by $V$ term to show whether that residual justifies the extra calls.
Why not use sample-and-vote self-consistency? Majority voting over several independent extractions reduces random errors. It cannot remove a systematic hallucination the model makes every time, and it spends a fixed number of samples per note. We include it at matched call budgets. The per-error-type analysis will show whether grounding failures persist under voting.
Proposed Experiments
The primary benchmark is the 2010 i2b2/VA concept and assertion task. We train the detector and tune $\lambda$ and $\mu$ on the training split, and report entity and assertion F1, average LLM calls per note, and wall-clock time per note. We then use i2b2 as an external set for a loop tuned on MIMIC-III notes, mirroring the Kiwi cross-institution protocol. The secondary benchmark is EHRCon, where the loop replaces CheckEHR's self-correction stage and we report recall against the 61.06% reference.
Three checks decide the proposal. First, the accept-all ablation must show measurable oscillation or F1 decay that the acceptance rule removes. Second, predicted budgets $V(x_0)/\epsilon$ must correlate with realized calls on held-out notes. Third, and decisive, final $V$ must rank-correlate with per-note error well enough that "$V = 0$" can serve as a label-free quality flag. We will report precision of that flag against gold errors.
If the third check fails, the energy is optimizing the wrong target and the approach should be abandoned or $V$ redesigned. If it passes, the loop provides what current clinical extraction pipelines lack: a termination guarantee, a per-note compute budget, and a routing signal for human review.
References
- Self-verification improves few-shot clinical information extraction
- EHRCon: Dataset for Checking Consistency between Unstructured Notes and Structured Tables in Electronic Health Records
- Information Extraction from Clinical Notes: Are We Ready to Switch to Large Language Models?
- Introducing mCODEGPT as a zero-shot information extraction from clinical free text data tool for cancer research
- Load balancing via random local search in closed and open systems
- Neural Lyapunov Control
- Sheaf-Guarded Updates: Streaming Structural Verification for Evolving Agent State
- Analytical Characterization of Epileptic Dynamics in a Bistable System
- Lyapunov spectra of chaotic recurrent neural networks
- Joint Entity Extraction and Assertion Detection for Clinical Text