When people ask a hard, multi-part question in conversation, they rarely state it all at once. They supply constraints piece by piece, wander off to ask something unrelated, and then come back and change a detail they gave five turns ago ("actually, make it the 2010 edition, not 2012"). Current language-model assistants store all of this as one flat transcript and re-reason over it every turn. That wastes compute. It also leads them to anchor on stale facts and lose track of the original goal. We propose a different design. The assistant keeps a persistent question-decomposition tree as its conversational state. The root is the user's overall goal, the leaves are small answerable sub-questions, and edges record which answers feed which. A stack of open trees handles digressions. When the user revises a constraint, only the sub-questions that depend on it are recomputed. When the user digresses, the unfinished tree is suspended and later resumed at the exact open branch.
In two sentences: treat the dialogue as the incremental construction and editing of a sub-question tree, rather than as a growing text log. Compute then goes only where the conversation actually changed something, or where a specific sub-answer is uncertain. The claim is testable. On multi-turn versions of multi-hop questions with scripted digressions and mid-dialogue revisions, the tree-as-state assistant should match the accuracy of re-deriving everything from scratch at a fraction of the tokens. It should also never silently change sub-answers that the revision did not touch.
Where this sits
The file points to four lines of work, and each supplies one piece while leaving a clear gap. RoHT supplies the method for trees. It builds a hierarchical decomposition tree, and a scheduler decides per node whether to query a knowledge base, read text, or decompose further. It reaches 46.45 exact match on KQA Pro and 54.4 on Musique, and its ablations show that flattening the tree into a chain hurts. But RoHT is single-turn: the tree is built once and thrown away. Patel et al. motivate decomposition in general, reporting roughly 24% F1 gains for GPT-3. They also constrain our design by documenting the "weakest link" problem, where one bad early sub-answer poisons everything downstream. That problem is why we need per-node checking rather than blind reuse of cached answers.
On the conversational side, Grosz and Sidner give the theoretical template. Discourse has an intentional structure of nested purposes plus a focus stack, a last-in-first-out record of what is currently being discussed. A digression pushes onto the stack and popping it returns to the suspended purpose. No LLM system in the file implements this over executable sub-questions. Two recent systems document the failure we target. ACR names contextual inertia, where models stick to early errors for the sake of narrative coherence, and state drift. Its fix is to rewrite the flat history with operators such as path pruning, which lifts HotpotQA multi-turn exact match from 21.7% to 35.1% at 278.5 tokens per turn. ACR still edits a flat log and cannot suspend and resume a specific branch. Conversational Tree Search navigates a dialogue tree and skips questions the user has already answered, but the tree is hand-authored and fixed in advance. We build the tree on the fly from the user's own query.
Adaptive depth and multi-agent checking enter as constraints on cost rather than as the headline. LASE supplies the gating technique. A leader answers first, and debate with adversarial supporters is triggered only when token-level confidence (minimum log-probability, maximum entropy) falls in an ambiguity zone. On GSM-Symbolic this gives 93.4% at 0.4k tokens versus 88.6% at 5.6k for standard debate. LASE gates whole answers on single-turn problems. It also fails when the model is confidently wrong, and it identifies this weakness itself. MARA routes each dialogue turn's draft to specialist refiners, reaching 74.51 on FoCus against 56.71 for no refinement and 47.07 for single-agent self-refinement. It works turn by turn on response drafts and has no notion of a suspended task. The overthinking results on TurtleBench constrain the design against defaulting to deep reasoning everywhere. There, o1-preview scored 84.40% against Claude 3.5 Sonnet's 89.49% at roughly 100 times GPT-4o's cost.
Core Claim
A dialogue state made of dependency-linked sub-questions plus a focus stack lets an assistant do three things a flat-history assistant cannot do cheaply.
The first is localized revision. When a user changes a constraint, the system finds the leaf that holds it and invalidates the nodes that consumed its answer, directly or transitively. It re-executes only those nodes. If $D(v)$ is the set of nodes downstream of the revised leaf $v$ and $c(u)$ is the cost of re-executing node $u$, the revision turn costs about $\sum_{u \in D(v)} c(u)$, rather than the cost of the whole tree plus re-reading the transcript.
The second is exact resumption. A digression pushes a fresh tree onto the stack. Popping it restores the parent tree with its open nodes intact, so previously established answers never need to be re-derived or restated.
The third is node-level adaptive depth. Each node is first answered in one direct pass. It escalates to a single adversarial checker, and then to further decomposition, only if its confidence is in LASE's ambiguity zone. To cover LASE's overconfidence blind spot, escalation is also forced whenever two sources disagree on the node (knowledge-base answer versus text answer), or whenever a node's upstream inputs changed.
The novelty is not trees, stacks or gating individually. It is making the executable decomposition tree the persistent object that conversation edits. Stability and cost savings under revision, and exact resumption after digression, are the measurable consequences.
Research Program
The system has three components.
The first is a tree editor. An LLM reads each new user turn and emits one structured operation against the current tree: add a constraint to a node, add a child sub-question, revise a leaf's value, push a digression, or pop back. It is trained by supervised fine-tuning on synthetic dialogues where the gold operation is known by construction (see below). It is prompted zero-shot as a baseline variant.
The second is a node executor following RoHT's scheduler. Its options are knowledge-base program, text reading, or decomposing further. It caches answers along with their confidence and source.
The third is a node-level verifier: a single adversarial checker in a star topology as in LASE, invoked only on escalation.
We deliberately keep the multi-agent part minimal, one checker and no all-to-all debate. LASE already shows symmetric debate behaves like majority voting at far higher cost.
The analysis centers on the tree editor, because editing errors are the new weakest link. A wrong "revise" operation silently corrupts cached answers. We will measure editor accuracy per operation type. We will also report how final-answer errors split between editor errors, executor errors, and verifier misses.
Benchmarks and construction
No benchmark in the file tests multi-hop questions delivered across turns with digressions and revisions. We therefore build one from KQA Pro and Musique, the benchmarks RoHT uses, so that the single-turn ceiling is known. Each gold multi-hop question with its decomposition is turned into a dialogue by a user simulator, in the style of ATPO's hidden-fact simulator (ATPO).
The simulator reveals constraints over several turns. With controlled probability it inserts a digression, which is an unrelated question drawn from the same pool and possibly itself multi-hop, possibly nested two deep. It also inserts a revision that changes one leaf constraint to a value with a different known gold answer, and it records which sub-answers should and should not change. We keep KQA Pro's setting where half the knowledge-base facts are removed and replaced with Wikipedia text, so that source disagreement actually occurs.
As a secondary check on realistic language, we use MultiWOZ 2.2, where users revise slot values across domains. For that setting we report joint goal accuracy; DS-DST's 51.7% on MultiWOZ 2.2 (Zang et al.) is the published reference point. We also use the multi-turn HotpotQA setting from ACR.
Baselines
The naive reference is a single strong instruction-tuned model given the full transcript each turn and asked to answer, with and without chain-of-thought. It shows how much contextual inertia and state drift our construction induces. The file has no reported result on Lacuna for this model on our constructed dialogues. We will run it on the same backbone and give it the same maximum context.
The strongest competitor on accuracy is stateless re-decomposition. At every turn, a rewriter condenses the transcript into one standalone question reflecting the latest constraints, and RoHT-style decomposition then runs from scratch. It shares our tree machinery and differs only in keeping no persistent state. So it isolates exactly what persistence buys. RoHT's published numbers (46.45 exact match on KQA Pro, 54.4 on Musique) set the single-turn ceiling that this baseline should approach if the rewriter is perfect.
The strongest competitor on efficiency is ACR, run with its published operators and training recipe on the same backbone. Its reported 35.1% exact match at 278.5 tokens per turn on multi-turn HotpotQA is the number to match on that setting.
The multi-agent competitor is LASE applied at the answer level each turn. Its published figures are 65.2% on GPQA-Diamond against 62.7% for majority voting, and 93.4% at 0.4k tokens on GSM-Symbolic. These do not transfer to our task, so we rerun it.
Two ablations isolate the key ingredients. The first keeps the tree and stack but recomputes the whole tree on revision, which isolates dependency-based invalidation. The second keeps everything but gates by confidence alone without the source-disagreement trigger, which isolates the overconfidence fix. All systems share one backbone, the same retrieval corpus and knowledge base, and a matched hyperparameter tuning budget on a held-out split. We report accuracy against tokens as a curve so that no method wins merely by spending more.
Beating the field means one of two outcomes. We match stateless re-decomposition's final exact match within noise while using clearly fewer tokens per turn and showing higher stability on unaffected sub-answers. Or we exceed it on post-revision and post-digression accuracy.
Why not just…?
Why not just rewrite the conversation into a standalone question and decompose from scratch each turn? This is the real threat. If the rewriter reliably absorbs revisions and ignores digressions, persistence buys only cost savings. On short dialogues those savings may be small. It is our primary baseline for that reason. We expect it to fail in two places: when revisions arrive after long digressions, where rewriters are exposed to the contextual inertia ACR documents, and on stability, since re-deriving everything lets unrelated sub-answers flip at random. If it matches us on both accuracy and stability, we will report that persistence is unnecessary for accuracy and recast the contribution as a cost result.
Why not just use a long-context model with the full transcript? ACR's result that fine-tuning on raw multi-turn context reaches only 21.7% exact match on HotpotQA, against 35.1% with active refactoring, suggests that longer context alone does not fix anchoring. Our naive baseline measures this directly. Its accuracy on revision turns, broken down by the number of turns since the original constraint, will show whether it degrades with distance.
Why not just use ACR's flat-history refactoring? ACR already prunes bad paths and filters noise cheaply. What it lacks is a representation of which answers depend on which constraint. After a revision, it must either keep the old conclusions or re-reason broadly. The head-to-head comparison on revision turns and on digressions nested two deep is where this shows up. If ACR matches us there, the tree adds nothing.
Why not just gate the whole answer with LASE each turn? Answer-level gating spends debate on the entire response when only one sub-question is uncertain, and it bypasses checking when the model is confidently wrong. The node-level against answer-level comparison at matched tokens, together with the confidence-only ablation on source-conflict cases, settles whether finer granularity and the non-confidence trigger matter.
Decisive Checks
Three results decide the proposal. First, on revision turns, the fraction of unaffected sub-answers that stay unchanged, together with the accuracy of affected ones, should favor the persistent tree over stateless re-decomposition. Second, the tokens spent on a revision turn should grow with the size of the invalidated subtree and not with the size of the full tree. We test this by varying tree depth while holding the revised leaf's depth fixed. Third, accuracy on the first question after popping a digression should stay flat as digression depth grows from zero to two, while flat-history baselines decline. If tree-editor errors turn out to dominate the error breakdown, the proposal's bottleneck is parsing dialogue into edits, not execution. That finding would redirect the work toward training the editor.
References
- Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answering
- Is a Question Decomposition Unit All We Need?
- Discourse Structure and the Proper Treatment of Interruptions
- ACR: Adaptive Context Refactoring via Context Refactoring Operators for Multi-Turn Dialogue
- Conversational Tree Search: A New Hybrid Dialog Task
- Efficient Multi-Agent Reasoning via Confidence-Guided Adaptive Debate
- Adaptive Multi-Agent Response Refinement in Conversational Systems
- TurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles
- ATPO: ADAPTIVE TREE POLICY OPTIMIZATION FOR MULTI-TURN MEDICAL DIALOGUE
- MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines