STATe-of-Thoughts: Structured Action Templates for Tree-of-Thoughts

Zachary Bamberger, Till R. Saenger, Gilad Morad, Ofra Amir, Brandon Stewart, Amir Feder

Abstract

Inference-Time-Compute (ITC) methods like Best-of-N and Tree-of-Thoughts are meant to produce output candidates that are both high-quality and diverse, but their use of high-temperature sampling often fails to achieve meaningful output diversity. Moreover, existing ITC methods offer limited control over how to perform reasoning, which in turn limits their explainability. We present STATe-of-Thoughts (STATe), an interpretable ITC method that searches over high-level reasoning patterns. STATe replaces stochastic sampling with discrete and interpretable textual interventions: a controller selects actions encoding high-level reasoning choices, a generator produces reasoning steps conditioned on those choices, and an evaluator scores candidates to guide search. This structured approach yields three main advantages. First, action-guided textual interventions produce greater response diversity than temperature-based sampling. Second, in a case study on argument generation, STATe's explicit action sequences capture interpretable features that are highly predictive of output quality. Third, estimating the association between performance and action choices allows us to identify promising yet unexplored regions of the action space and steer generation directly toward them. Together, these results establish STATe as a practical framework for generating high-quality, diverse, and interpretable text.

Introduction: Moving Beyond Randomness

Current Large Language Models (LLMs) often use Inference-Time-Compute (ITC) strategies like Tree-of-Thoughts (ToT) to improve reasoning. To generate diverse outputs, these methods typically rely on high-temperature sampling—essentially adding randomness to the token selection process. However, this approach often results in "more of the same" or degraded quality without providing meaningful semantic variety. Furthermore, the "black box" nature of these reasoning paths makes it difficult to control rhetorical strategies or analyze which patterns lead to the best results.

The STATe Framework

STATe-of-Thoughts (STATe) transforms the reasoning process into a structured Plan → Generate → Evaluate → Select loop. Instead of stochastic branching, it uses discrete, interpretable Action Templates.

The STATe-of-Thoughts workflow: 1) Defining action templates, 2) Generating outputs via a structured tree, and 3) Evaluating features to steer future generation.

The framework consists of several modular components:

  • Action Templates: Discrete "tools" that encode high-level reasoning choices, split into Structure (e.g., Conditional, Concession) and Content (e.g., Risk & Consequences, Feasibility).
  • The Controller: An LLM that selects which actions to explore at each step of the reasoning tree.
  • The Generator: Executes the action by prefilling the model’s response with a specific prefix (e.g., "However..."), forcing the model to follow the chosen strategy.
  • The Evaluator: Uses Process Reward Models (PRMs) and Outcome Reward Models (ORMs) to guide the search toward high-quality branches.
  • Synthesis Modes: Once a path is complete, STATe uses synthesis prompts (e.g., "Strict" or "Faithful") to weave intermediate thoughts into a final, coherent response.

Case Study: Argument Generation

To evaluate the framework, we conducted a case study on argument generation regarding a ban on single-use plastics. This task allowed us to test how well explicit action sequences could predict and steer argument quality.

A detailed view of the argument generation process where the controller selects specific reasoning paths to produce distinct outputs.

Key Insights and Results

  • Superior Diversity: On the NoveltyBench benchmark, STATe produced up to twice as many semantically distinct outputs as traditional temperature-based sampling across seven different LLMs.
  • Interpretable Attribution: Because STATe logs every action, researchers can use regularized regression (LASSO) to identify winning patterns. For instance, we found that pairing "Exemplification" with "Justice and Fairness" content lenses was particularly effective.
  • Targeted Steering: By forcing the model into high-scoring action sequences identified by the regression model, we generated "targeted" arguments that achieved a 77–81% win rate against random exploration baselines.
  • Efficiency: The inclusion of a "FINISH" action allows the controller to stop reasoning once it has enough information, preventing the "overthinking" often seen in other ITC methods.

Discussion

STATe demonstrates that explicitly steering a model to "be contrarian" or "use a precedent" is far more effective than relying on randomness. While the current implementation relies on text prefilling—a technique most easily used with open-source models—it establishes a practical framework for generating high-quality, diverse, and interpretable text. Future work will focus on automating action template engineering and exploring multi-turn conversational dynamics.

Framework available at github.com/zbambergerNLP/state-of-thoughts.

Summary updated

Abstract

Inference-Time-Compute (ITC) methods like Best-of-N and Tree-of-Thoughts are meant to produce output candidates that are both high-quality and diverse, but their use of high-temperature sampling often fails to achieve meaningful output diversity. Moreover, existing ITC methods offer limited control over how to perform reasoning, which in turn limits their explainability. We present STATe-of-Thoughts (STATe), an interpretable ITC method that searches over high-level reasoning patterns. STATe replaces stochastic sampling with discrete and interpretable textual interventions: a controller selects actions encoding high-level reasoning choices, a generator produces reasoning steps conditioned on those choices, and an evaluator scores candidates to guide search. This structured approach yields three main advantages. First, action-guided textual interventions produce greater response diversity than temperature-based sampling. Second, in a case study on argument generation, STATe's explicit action sequences capture interpretable features that are highly predictive of output quality. Third, estimating the association between performance and action choices allows us to identify promising yet unexplored regions of the action space and steer generation directly toward them. Together, these results establish STATe as a practical framework for generating high-quality, diverse, and interpretable text. Our framework is available at https://github.com/zbambergerNLP/state-of-thoughts.
BibTeX
@inproceedings{bamberger2026state,
  author = {Zachary Bamberger and Till R. Saenger and Gilad Morad and Ofra Amir and Brandon Stewart and Amir Feder},
  title = {{STATe-of-Thoughts: Structured Action Templates for Tree-of-Thoughts}},
  booktitle = {WAB @ COLM},
  year = {2026},
  url = {https://openreview.net/forum?id=tvHeYgykXS},
}
Figure