When a language model answers a multi-step question, we can split the question into a tree of smaller sub-questions. We can then let the model answer any part of that tree in one call or in several. Recent work shows that a model is more accurate when its answers agree across these choices of how much of the tree to merge into one call. That leaves two problems open. Nobody has said which merges should be trusted. And nobody knows how to search the exponentially many possible trees for a good one. This proposal makes one claim about each. A merge is safe when the sub-questions being merged are connected, meaning each one shares an entity or an answer slot with the others. Restricting the search to connected trees is also what makes that search affordable.
The idea borrows a pattern that appears independently in two literatures. In stable homotopy theory, simplifying a ring-like object by "localizing" it keeps its multiplication intact only when the object is connective, that is, has nothing in negative dimensions for errors to leak into. In database and constraint solving, forcing each piece of a decomposition to share variables with its neighbours prevents the blow-up from combining unrelated pieces. We treat this shared pattern as a hypothesis generator, not a theorem: connectivity is the condition under which simplification preserves structure. We then test it where it is cheap to test and useful if true, namely in how language models answer decomposed questions.
Core Claim
Operads for compositional reasoning in LLMs introduces three ideas. It formalizes question templates with blanks as an operad, an algebraic object whose operations are plugged into each other's inputs. It treats a model as an "algebra" that evaluates such composites. It defines operadic consistency as agreement between evaluating the full tree step by step and evaluating any partial collapse of it, where a collapse merges a subtree into a single prompt. Across eight models and four benchmarks, this consistency correlates with accuracy at Pearson r between 0.70 and 0.98. Temperature-based self-consistency shows no significant correlation. The paper works only with human-written trees. It never generates or selects decompositions, and its suggestion that topological invariants could classify failures stays informal.
Our claim has two parts. The first is diagnostic. Inconsistency between a subtree and its collapse is concentrated in collapses whose merged sub-questions are disconnected. In such a collapse, two sub-questions share no entity and no answer slot, so the merged prompt asks the model to carry independent reasoning threads at once. This is the language-model analogue of the Cartesian products that Soft and Constrained Hypertree Width eliminates with its Connected Cover (ConCov) constraint.
The second part is algorithmic. Choosing the most consistent decomposition is a combinatorial search, related to the NP-hard decomposition problems catalogued by A Comparison of Structural CSP Decomposition Methods and Optimal decomposition of belief networks. We claim that restricting this search to connected trees and connected collapses does two things. It shrinks the search enough that exact backtracking with reuse of repeated sub-questions fits within a fixed model-call budget. And the connected tree that maximizes consistency is more accurate than both the human-written tree and the best tree found without the constraint.
The second part could fail even if the first holds. Connectivity might cost something, just as Backtracking Procedures for Hypertree, HyperSpread and Connected Hypertree Decomposition of CSPs leaves open whether requiring connected decompositions ever forces larger width. We measure that cost directly as extra steps.
Research Program
Formalization. We define a decomposition of a question $q$ as a tree of sub-question templates in which every edge fills one blank. For a set $S$ of tree nodes merged into one prompt, $S$ is connected when the graph is connected whose vertices are the members of $S$ and whose edges join two sub-questions sharing an answer slot or a named entity. A decomposition is connected when every allowed collapse is connected. We define the consistency score of a tree $T$ as the fraction of allowed collapses $C$ whose answer matches the stepwise answer:
$$\mathrm{cons}(T) = \frac{1}{|\mathcal{C}(T)|}\sum_{C \in \mathcal{C}(T)} \mathbf{1}\big[\mathrm{ans}(C) = \mathrm{ans}(T)\big]$$
This mirrors Bottman and Richardson's measure, with $\mathcal{C}(T)$ restricted by connectivity in the constrained variant.
Analogy and its limits. The homotopy-theory results motivate the hypothesis and also bound what we claim. Homotopical localizations of module spectra shows that a localization that does not commute with suspension still preserves ring and module structure, provided the spectrum is connective. Its counterexamples show collapse otherwise. Localization of algebras over coloured operads shows that algebras over well-behaved coloured operads keep their structure under localization. We do not claim a formal functor from question trees to spectra. We borrow only the testable prediction: structure-preserving simplification needs a connectivity guard, and failures should cluster where the guard is absent.
Search. We generate candidate sub-questions with the model itself. We then run a backtracking search over connected trees in the style of Subbarayan and Andersen's exact solver. Their solver caches structurally identical sub-hypergraphs; our search caches identical sub-questions so that each is answered once. The search prunes any partial tree whose observed collapses already disagree. We stop at a fixed call budget and return the tree with the highest consistency score. The unconstrained variant uses the same procedure without the connectivity filter.
Baselines
The naive reference point is answering the question directly in one call. A second reference evaluates the benchmark's human-written tree step by step, which is exactly the setting in Operads for compositional reasoning in LLMs. That paper reports correlations, not accuracies: r between 0.70 and 0.98 between consistency and accuracy, and no significant correlation for temperature self-consistency at K=3 and K=10. For the accuracy of human-tree evaluation on these benchmarks, no reported result was found on Lacuna, so we measure all accuracies ourselves.
Strongest competitor: unconstrained consistency-maximizing search. This baseline uses the same generator, the same scoring function and the same call budget as our method, but does not filter for connectivity. It is the strongest competitor because it already uses the one signal known to track accuracy. It therefore isolates whether connectivity adds anything.
Self-consistency voting. We run self-consistency voting over freely generated decompositions with a matched number of calls. This is the standard alternative and has a documented weakness, the absent correlation reported by Bottman and Richardson.
Minimal-steps selection. We pick the decomposition with the fewest sub-questions. This is the analogue of choosing minimal width, which Lanzinger et al. show can be slower than a slightly wider connected decomposition.
Random-connectivity ablation. For each connected tree, we construct a matched tree of equal size that contains exactly one disconnected collapse. Any accuracy gap can then be attributed to connectivity rather than to tree size.
All methods use the same models, the same prompts per step, and the same tuning on a shared development split. Our method beats the strongest competitor if it shows higher exact-match accuracy at an equal call budget, with a paired bootstrap interval excluding zero on at least three of four benchmarks.
Why not just…?
Why not just let the model write its own decomposition and vote over samples? Bottman and Richardson report that temperature-based self-consistency has no significant correlation with accuracy, while collapse-based consistency does. Voting is therefore a weak selector. We still include it with a matched budget so the gap is measured rather than assumed.
Why not just pick the shortest decomposition? Fewer steps means fewer calls. But Soft and Constrained Hypertree Width found 2x to 10x query speedups from decompositions chosen by cost and connectivity rather than minimal width. The language-model analogue predicts that short trees often hide disconnected merges. The minimal-steps baseline tests this directly, and if it matches our method the connectivity story is wrong for this setting.
Why not just maximize consistency and drop the connectivity constraint? This is the real threat. Consistency alone may already avoid bad merges, because disconnected merges would show up as inconsistent and get penalized. If so, connectivity is only a speed-up and not an accuracy gain. The unconstrained-search baseline and the random-connectivity ablation settle the question. We will report the result either way. If the constraint helps only by enlarging the search within budget, we will say that.
Why not just skip the homotopy theory? The method needs none of it to run. Its role is to predict where failures concentrate before we look. If the stratified analysis below finds failures spread evenly across connected and disconnected collapses, the analogy has no predictive value and we drop it.
Proposed Experiments
Diagnostic test (H1). We use the four benchmarks from Bottman and Richardson: HotpotQA and DROP with QDMR decomposition annotations from the Break dataset, plus MuSiQue and StrategyQA with their native trees. We enumerate all collapses of each human tree, label each collapse connected or disconnected using slot references and entity overlap, and compare inconsistency rates across the two strata. We also compare how well consistency computed only over connected collapses predicts per-question errors, measured by AUROC, against consistency over all collapses. This requires no search and is cheap. A null result here stops the program.
Search test (H2). We run all baselines at three call budgets on open models across the parameter range used in the original study. We report accuracy, consistency, calls per question, search-space size with and without the constraint, and the extra steps that connectivity costs relative to the shortest tree. That last quantity is the empirical counterpart of Subbarayan and Andersen's open question about whether connected hypertree width can exceed standard hypertree width.
Controlled test. We generate multi-hop questions from cyclic query patterns over the Hetionet biomedical knowledge graph, as used by Lanzinger et al., with answers computed exactly. Connected and disconnected decompositions can then be constructed deliberately, so the ablation does not depend on noisy entity matching.
Decisive check. The program succeeds if two things hold. Disconnected collapses must account for a clearly disproportionate share of inconsistency on human trees. And the connected search must beat unconstrained search at equal budget on at least three of four benchmarks and on the controlled Hetionet questions.
References
- Operads for compositional reasoning in LLMs
- Soft and Constrained Hypertree Width
- A Comparison of Structural CSP Decomposition Methods
- Optimal decomposition of belief networks
- Backtracking Procedures for Hypertree, HyperSpread and Connected Hypertree Decomposition of CSPs
- Homotopical localizations of module spectra
- Localization of algebras over coloured operads