Guarded Exits: Statechart Control Against Hallucination Snowballs in Multi-Hop QA

When a language model answers a question that needs several reasoning steps, a single unsupported intermediate "fact" tends to become a permanent premise. Every later step then builds on it, and the final answer is fluent, confident and wrong. Recent prompting methods break multi-hop questions into a loop of small sub-tasks and report large gains. Their loop is flat, though, and the model itself decides when to move on. Nothing in it can say "this sub-answer has no evidence, so cancel everything that depended on it and nothing else." We propose running multi-hop reasoning as a statechart. A statechart is a state machine whose states can contain sub-machines and whose parallel branches run side by side. Each sub-question executes inside its own nested state with its own private scratch context. It can leave that state only through a guarded exit, meaning it must hand back a fact tied to a quoted passage that ordinary code can check. When the check fails, a single parent-level event cancels exactly the sub-tree that depended on that fact. The system then either tries a different decomposition or passes an explicit "unsupported" result upward instead of inventing an answer.

The bet is that hallucination in multi-hop question answering is largely a control-flow problem rather than a knowledge problem. Statecharts were built to solve exactly this kind of control problem: scoped contexts, preemption from the parent level, and clean exit semantics. If the bet is right, the gains should appear where flat loops are weakest: on deeper and branching questions, on questions whose evidence has been deliberately removed, and on small models that cannot reliably follow a loop's transition rules.

Core Claim

Moving transition control out of the language model and into a deterministic hierarchical executor should reduce baseless intermediate facts and fabricated final answers without lowering answer F1. The executor uses nested states with private context, guarded exits that require verifiable evidence, and preemption at the right level of the hierarchy. The specific new ingredient is localized rollback. When a hop fails its guard, the executor knows from the hierarchy which ancestor sub-questions depend on it. It invalidates only those, keeps sibling branches that succeeded, and never writes the unsupported fact into any context that later steps will read.

We predict the following:

  • The advantage over a flat loop that has the same per-hop verifier should be near zero on 2-hop questions.
  • The advantage should grow on 3- and 4-hop and branching questions, because only there does the dependency structure carry information a flat loop lacks.
  • If that prediction fails, the hierarchy is not doing the work, and we will say so.

Relation to Prior Work

Two papers motivate the proposal and supply its starting design.

  • FSM and SG-FSM turn multi-hop question answering into a fixed cycle of Decomposer, Searcher, Revisor and Judge/Terminator roles. They report MuSiQue F1 around 40.5–48.5 and full format compliance.
  • Both use a single-layer automaton. A sub-question that itself needs decomposition cannot open a nested machine.
  • Transitions are decided by the model's own textual self-evaluation.
  • The Revisor rewrites the original question with each discovered fact, which commits that fact before it has been checked against anything.
  • SG-FSM also reports difficulty with 7B models that do not keep to the transition contract.
  • We keep their narrow per-role prompts but make the hierarchy and the transition guards explicit and executable.

Two lines of work show that hierarchy helps but do not supply formal state semantics.

  • HyMem isolates planner, executor and reasoner contexts in long-horizon agents. It triggers memory folding with turn-count heuristics, and its isolation is set by agent boundaries rather than by state entry and exit.
  • MATA routes among visual tool agents with a trained hierarchical automaton and an append-only memory that supports backtracking. It routes tasks; it does not track which factual commitments depend on which evidence.
  • We borrow HyMem's isolation principle and MATA's auditable transition log. We attach both to the statechart's own semantics: entering a state creates its scratch context, and exiting it discards that context and returns one typed fact.

Three papers constrain the design and also act as competitors.

  • RAGTruth shows that baseless additions outnumber explicit contradictions and that errors pile up toward the end of responses. This is why our guard checks for missing support, not only for contradiction.
  • RT4CHART verifies atomic claims in a local-then-global hierarchy, but only after generation has finished. We move a lightweight form of that check onto each state exit, so verification happens before a fact can spread.
  • KCTS intervenes during decoding at the token level, at high latency and with no notion of reasoning phases. Our intervention point is much coarser and cheaper: one check per sub-question rather than per token.

The statechart semantics themselves come from Harel's work, Statecharts: A Visual Formalism for Complex Systems and Some Thoughts on Statecharts, 13 Years Later. We use their OR-nesting, AND-regions and parent-level exit transitions. The CARA infusion pump model in Formal specifications and analysis of the CARA Infusion Pump Control System is the direct analogy. There, one exit transition on the Active macro-state handles a global fault without an arc from every micro-state. Here, one "unsupported" event on a sub-question's macro-state cancels its whole sub-tree.

Research Program

The executor. We build a small statechart interpreter in ordinary code. The language model never chooses transitions; it only fills the content of a state.

  • Decompose. The top-level state asks the model to split the question into sub-questions and to label their dependencies, for example "Q2 uses the answer of Q1."
  • Nesting and parallel regions. Each sub-question becomes a nested state. Sub-questions that do not depend on each other become parallel regions, for example the two arms of a "which is older, X or Y" comparison. Each region gets a join guard that waits until both arms have returned supported facts.
  • Inside a sub-question state. The model works with a private context. That context holds the parent's question, the facts returned by the states it depends on, and the candidate paragraphs. It never contains sibling branches' scratch work or earlier failed attempts.
  • Recursion. If a sub-question is itself compositional, the state can open a further nested decomposition. We cap the depth at three.

The guarded exit. A sub-question state may exit only by returning three things: an answer string, a verbatim quoted span, and the identifier of the paragraph the span comes from. The guard has two parts.

  • Quote check. Plain code confirms that the span actually occurs in that paragraph.
  • Support check. A small entailment checker confirms that the span supports the answer to the sub-question. We will build this checker by fine-tuning a 7B–13B model in the style of the RAGTruth detector, and we will train it on data disjoint from the evaluation questions.

If the guard fails, the executor raises an unsupported event at the sub-question's macro-state, which cancels all of its nested children. The parent then does one of two things:

  1. It re-enters with a different decomposition, starting from a clean context with no memory of the failed attempt, up to a fixed retry budget.
  2. It marks the sub-question unsupported and propagates that status upward. Any ancestor whose answer depends on it becomes unsupported as well, while independent siblings keep their verified facts.

The final answer is produced only when the root state's guard holds. Otherwise the system abstains and reports which sub-question lacked evidence.

What we analyze. We log every transition, which gives a dependency-annotated trace for each question. From these traces we measure:

  • how often a fact that failed its guard would have reached the final answer in a flat loop;
  • at what depth failures happen;
  • whether retries recover answers or only add cost.

Baselines

Naive references. The do-nothing reference is direct prompting and standard chain-of-thought on the same backbone with the same paragraphs. This shows the base rate of fabrication.

Strongest published method. The strongest competitor is SG-FSM2, which runs its flat four-role loop and then a final Summarizer that reviews the whole chain. SG-FSM2 is the most direct prior use of automata against hallucination in this exact task. It reports MuSiQue F1 of 40.5 with GPT-3.5 and 48.5 with Qwen (SG-FSM). We will re-implement it with its published prompts on the same backbones and paragraph sets rather than rely on the reported numbers. Beating it means two things together: higher answer F1 on MuSiQue at matched cost, and a clearly lower rate of unsupported final answers.

Post-hoc audit and resample. This baseline lets SG-FSM or chain-of-thought produce a full trace, audits it with an RT4CHART-style claim verifier using the same entailment checker, and resamples when the audit fails. RAGTruth shows that filtering with a detector cut a Llama-2-7B model's hallucination rate by 63.2% (RAGTruth). That is the main evidence that verification without control-flow changes is already strong.

Self-consistency. Majority voting over sampled chain-of-thought traces, with the number of samples chosen to match our total budget.

Fairness rules. All methods get the same paragraphs and the same entailment checker wherever one is used. Compute is matched by the total number of language-model calls and generated tokens per question. Every method gets the same prompt-tuning budget, measured as the same number of development-set iterations.

Ablations. These isolate the key ingredient:

  • (a) Flat loop plus per-hop guard. SG-FSM with our exit guard placed after every Searcher step and plain restart-from-scratch on failure. This tests whether the guard alone explains the gains.
  • (b) Nesting without guards. This tests whether scoped contexts alone help.
  • (c) Guards plus nesting, global restart. Full guards and nesting, but any failure restarts the whole question instead of rolling back only the dependent sub-tree.

The proposal's distinctive claim holds only if the full system beats (a) and (c) on 3–4-hop questions.

Why not just…?

Why not just add a per-hop verifier to the existing flat loop? This is the real threat, and ablation (a) is built to settle it. On 2-hop HotpotQA and 2WikiMultiHopQA we expect the two to be roughly equal. Both FSM papers already report being "on par" with baselines on those easier sets, and a two-step chain has little dependency structure for a hierarchy to exploit. Our claim is confined to MuSiQue's 3- and 4-hop compositions, where a flat loop cannot tell which earlier commitments a failed hop invalidates. Without that information it has two options: restart, which costs calls and discards verified siblings, or continue, which snowballs. If (a) matches the full system on 3–4-hop MuSiQue at equal cost, we will drop the hierarchy claim and report the guard as the contribution.

Why not just audit after generation and filter or resample? Post-hoc auditing is strong on detection: RT4CHART reports the best answer-level F1 on RAGTruth++ and RAGTruth-Enhance at about $0.0104 per sample (RT4CHART). However, a failed audit tells you to throw away the whole trace. Resampling then pays again for the hops that were already correct, and it never produces a localized abstention. The audit-and-resample baseline at matched calls tests directly whether in-process guarding is worth its complexity. If the two tie in both accuracy and cost, in-process guarding is not justified.

Why not just sample several chains and vote? Voting reduces variance but not systematic fabrication. When the gold evidence for a hop is missing, every sample is pushed to invent the same bridging fact, and the vote agrees on it. The evidence-removal test below measures this directly. No reported result was found on Lacuna for self-consistency on MuSiQue, so we will run it ourselves.

Why not just use HyMem-style context isolation? HyMem's gains on GAIA and Browsecomp-plus come from isolating noisy tool traces in long agent runs (HyMem). Isolation alone does not stop a wrong distilled fact from being returned to the planner. Ablation (b), nesting without guards, is effectively HyMem's principle applied to multi-hop question answering. We expect it to help a little on its own and much less than the guarded version.

Proposed Experiments

Datasets and backbones. The main testbed is MuSiQue-Ans (MuSiQue), split by hop count, with HotpotQA and 2WikiMultiHopQA as controls where we expect no gap. We will use Qwen and GPT-3.5-class backbones as in SG-FSM, plus a 7B open model. The 7B model tests whether moving transition control into code removes the small-model adherence failure that SG-FSM reports.

Metrics. We report:

  • answer EM and F1, and supporting-paragraph F1;
  • baseless intermediate rate: the share of intermediate facts not entailed by any paragraph, judged by a held-out verifier and checked by human annotation on a sample of 300 traces;
  • calls and tokens per question.

MuSiQue's human ceiling of 78.0 answer F1 and 93.9 support F1 frames how much room remains.

Evidence-removal test. For each MuSiQue question we delete the gold paragraph for one chosen hop, at depth 1, 2 or 3. The correct behavior is then to abstain, ideally while naming the failed sub-question. We measure:

  • fabrication rate: answers given with no remaining support;
  • abstention precision and recall: precision on these edited questions, and recall measured as no loss of answers on the unedited ones;
  • localization accuracy: whether the system names the hop whose paragraph was removed.

We will also use MuSiQue's released unanswerable contrast questions where they are available. No reported result was found on Lacuna for any automaton method on those questions.

Decisive checks. Four outcomes decide the claims:

  • If the full system beats ablation (a) by a meaningful margin on 3–4-hop MuSiQue and on depth-2/3 evidence removal, while showing no gap on 2-hop sets, localized rollback through the hierarchy is the operative ingredient.
  • If the full system's fabrication rate under evidence removal is not lower than audit-and-resample at matched calls, the in-process claim fails.
  • If the 7B backbone under our executor does not close a substantial part of its gap to the larger model relative to SG-FSM, the "control out of the model" argument fails for small models.
  • As a transfer check, we will run long answers on the RAGTruth question-answering portion with each section treated as a nested state. We will test whether the reported rise in errors toward the end of responses flattens, and we will compare against detector filtering on the same responses.