Redirecting Stale Identifiers During Decoding Under Schema Drift

Databases and APIs change. Columns get renamed, tables are split, and fields are deprecated. A language model that writes SQL learned the old names during training. It also tends to see them again at inference time, in retrieved example queries, old documentation, and conversation history. The obvious fix is to mask the decoder so it can only produce identifiers that exist in the current schema. Masking means zeroing out the probability of any token that would spell an invalid name. We argue this fix has a hidden failure.

Identifiers are split into subword fragments. Old and new names often share a prefix: customer_id was renamed client_id, but customer_name still exists. So a model that "wants" the stale name gets steered by the mask onto whichever valid name its partial spelling still matches. The query parses, executes, and silently returns the wrong answer. That converts loud errors, which an execution check can catch, into silent ones it cannot.

We propose a different decoder. It recognizes when the model is partway through spelling a deprecated identifier and replaces it with that identifier's recorded successor. The successor comes from a lineage graph built from the schema's migration history. Our claim is that this lineage-aware redirection, not plain masking, is the right way to inject knowledge updates at the token level.

Core Claim

Under schema drift, a meaningful share of text-to-SQL errors are stale-identifier errors: the model emits a name that used to be correct. Hard masks over the current vocabulary of names do not fix these errors. They relocate many of them into wrong-but-valid identifiers. A decoder that tracks both current and retired names can catch a retired name as the model commits to it. It then rewinds generation to where that name began and forces the successor name in its place. This should recover most stale-identifier errors without lowering the silent-error rate, at lower cost than post-hoc agent correction loops. When a column has split into several successors, the decoder restricts the choice to those successors. When a column was deprecated with no successor, it forces an explicit failure.

Where This Sits in Prior Work

Methods we reuse. Two lines of work supply the mechanics.

  • Grammar-constrained decoding handles the mismatch between subword tokens and grammar symbols. Poesia et al. observed that tokens rarely align with program identifiers. Ugare et al. buffer the unfinished character remainder of a partial identifier. GREATGRAMMA compiles a table, offline, that maps each token to the grammar symbols it can produce, with 5–32 ms per-token overhead.
  • IterGen tracks where each grammar symbol starts. It can roll the model's cached attention state back to a symbol boundary and check generated SQL columns against an external schema.

The closest existing work. IterGen comes nearest to our proposal. It was evaluated only on static schemas. When a check fails, it backtracks and penalizes the rejected tokens, then resamples. It has no notion of which valid name the rejected one should become. That distinction is the whole point here. Resampling under a penalty is exactly where prefix collisions produce silent substitutions.

Evidence that motivates the problem. Several papers show that simply supplying fresh information does not stop models from using stale facts.

  • StateMem finds that models anchor on salient superseded values even under oracle retrieval, and that chain-of-thought can make this worse.
  • SituatedQA reports models 15 points less accurate on updated facts than on stable ones, even with an updated corpus.
  • RealTime QA shows models falling back to parametric knowledge when retrieval is insufficient.

These are the reasons we enforce updates in the decoder rather than in the prompt.

The data structure we borrow. StateMem's explicit derived_from dependency edges inspire our lineage graph. StateMem uses those edges to assemble prompt context. We use them to drive token-level rewrites.

Work that operates elsewhere. AURORA adapts retrieval indexes to drift but never touches generation. Zhang & Choi predict how long a fact stays valid at the sentence level, not at the level of individual identifiers.

Multi-agent systems. Our multi-agent comparison point is SQL-of-Thought. It decomposes text-to-SQL into schema-linking, planning, and correction agents and repairs queries after they execute. We do not add agents. We test whether a decoder-level constraint can absorb the schema-error portion of that correction loop. We also test whether it can catch errors that the loop structurally cannot see, because those queries execute without complaint.

Research Program

Lineage graph. For each database, we maintain a graph of identifier versions. A schema steward builds it by retrieving and parsing migration scripts and changelogs. The steward can be a simple parser or one LLM call, and its accuracy is reported separately. Each retired name links to zero, one, or several successors, each with an effective date.

Dual lexicon. The decoder's identifier vocabulary is the union of current and retired names, compiled into a prefix structure over the model's subword tokens. The structure is recompiled incrementally when the graph changes, and we measure how long that takes.

Decoding rule. Retired names are not masked. The model is allowed to spell them, because forbidding them early is what produces collisions. At the first point where the character buffer is consistent only with retired names, or at the identifier's boundary if it stays ambiguous until then, the decoder takes three steps.

  1. It rewinds the cached state to where the identifier started.
  2. It constrains the next terminal to the successor set.
  3. It continues generating.

If the retired name has no successor, the decoder emits an explicit "deprecated reference" marker rather than guessing. We will compare the two trigger points: the earliest unambiguous point versus the identifier boundary.

Analysis. The analysis splits outcomes into four classes:

  • correct;
  • loud failures, where the query does not parse or execute;
  • silent failures, where the query executes but returns wrong results;
  • silent failures caused specifically by prefix collision.

For the last class, we compare against an unconstrained run of the same model to see which name it was originally spelling.

Benchmarks and Construction

No schema-drift text-to-SQL benchmark appears in our sources, so we construct one from Spider and its variants, Spider-Realistic and Spider-SYN, as used by IterGen and SQL-of-Thought.

Migrations. For each Spider database we apply scripted migrations of several types:

  • renames chosen to share a subword prefix with a surviving column;
  • renames with no shared prefix;
  • splits of one column into several;
  • table merges;
  • deprecations with no successor.

Gold queries are rewritten mechanically and checked by execution against the migrated database.

Stale context. Few-shot examples and a retrieved documentation snippet keep the old names. This reproduces the situation where retrieval itself carries obsolete knowledge. Because Spider schemas are likely memorized from pretraining, the models also carry stale parametric knowledge, which is the regime we care about.

Collision subset. We report results separately on queries whose gold answer touches a renamed identifier with prefix collision. This is where the hypothesis lives.

Models and limits. We use open models in the Qwen2.5 and Llama-3 families so that decoding can be controlled at the token level. A secondary check uses StateMemBench in a narrow way: we test only whether lineage-aware enforcement of values helps in structured outputs. We will say plainly that the synthetic migrations are the main threat to external validity.

Baselines

Naive reference. The model receives the current schema in its prompt, along with the stale examples and documentation, and decodes without any constraints. This measures how often stale names appear at all. If that rate is negligible, the project is moot and we stop.

Masked grammar decoding. This is GREATGRAMMA-style SQL grammar decoding GREATGRAMMA with an identifier vocabulary restricted to the current schema. It is the "obvious" approach and the one our silent-failure argument targets. It runs on the same models, prompts, and sampling settings. Beating it means higher execution accuracy on the collision subset and a lower silent-failure rate, with comparable overall accuracy elsewhere.

IterGen with a live schema check. We run IterGen with its column-existence check pointed at the migrated database. This is the strongest token-level competitor. IterGen reports Spider execution accuracy improving from 32.7% to 50.7% for Qwen2.5-1.5B-Instruct over its baseline, and an 18.5% mean improvement over SynCode. We run it with the same model and the same backtracking budget. Beating it means the gain comes from successor information rather than from backtracking itself.

SQL-of-Thought. We run SQL-of-Thought with its correction loop and live execution feedback. It reports 91.59% execution accuracy on Spider and 90.16% on Spider-Realistic, but on static schemas, so those numbers are not directly comparable. We re-run it with the same backbone model on our migrated data. We report both accuracy and total tokens per query. A fair win means matching its accuracy at a fraction of the tokens, or beating it on silent failures at equal tokens. We will also run it with our decoder underneath to test whether the two combine.

Ablations. Four ablations isolate the key ingredient:

  1. The lineage graph placed only in the prompt, as a StateMemWrapper-style instruction to trace the value chain and then answer StateMem.
  2. Redirection to a random current identifier.
  3. Masking retired names from the very first token.
  4. The two trigger points described above.

Comparing ablation 1 with the full method separates "knowing the lineage" from "enforcing it." Comparing ablation 3 with the full method tests whether early masking is what causes collisions.

Why not just…?

Why not just put the migration log in the prompt? This is the most serious threat, because it is cheap and model-agnostic. Evidence against it is indirect. StateMem shows salience anchoring persists even with oracle context, and reasoning traces can rationalize outdated values. Those findings concern conversational state, not SQL identifiers. The prompt-only lineage ablation settles the question directly. If it matches redirection on the collision subset, the token-level machinery is not justified and we will say so.

Why not just let an agent fix it after execution? SQL-of-Thought attributes a 10% gain to its guided correction loop. However, its error signal comes from execution. A query that hits a colliding but valid column runs fine and is never flagged. Our silent-failure breakdown measures exactly this blind spot. The token comparison measures the cost.

Why not just mask to current names, as grammar decoders already can? On drift with no shared prefixes, masking may work as well as redirection, and we expect to find that. The proposal stands or falls on whether prefix-colliding renames produce a measurable silent-failure rate under masking. The collision subset is designed to show this. If that rate is small, the simpler method wins.

Why not just fine-tune on the new schema? That requires retraining at every migration and does not remove stale context retrieved at inference time. AURORA makes the analogous cost argument for indexes, reporting a 182x speedup over retraining. We include a small fine-tuned reference on a subset of databases to quantify the gap, rather than assuming it.

Decisive Checks

The central experiment is restricted to queries where the unconstrained model emits a retired identifier. On those queries, we compare the share that become correct, loudly wrong, or silently wrong under four methods: masking, IterGen, prompt-only lineage, and redirection.

The hypothesis survives if two things hold. First, masking shows a clearly higher silent-wrong share than redirection on prefix-colliding renames. Second, redirection beats prompt-only lineage on that subset.

We also report the recompilation latency of the dual lexicon after each migration. We report how accurately the steward extracts lineage, and how errors in that extraction propagate into the final queries.