Abstract
Large Language Models (LLMs), when combined with agent mechanisms, show great promise in applications requiring robust planning ability, such as financial analysis and medical diagnostics. However, the increasingly complex reasoning structures designed to enhance the planning ability of language agents often exceed the processing and comprehension capabilities of LLMs, thereby limiting their effectiveness. To address these challenges, we introduce the Encoder of Thoughts (EoT), a novel reasoning structure modeling method based on graph neural networks. EoT processes the reasoning structures of planning methods through a plug-and-play structural encoder and aligns these structural information with the input space of LLMs, enabling seamless integration with existing language agents. Experiments on multi-step reasoning and plan generation demonstrate that EoT significantly improves the performance of language agents. Moreover, EoT demonstrated stable results when combined with different LLMs and planning algorithms, further underscoring its potential for broader application.
Introduction: The Comprehension Bottleneck
As Large Language Models (LLMs) are increasingly deployed for complex problem-solving, the paradigm has shifted from simple Chain-of-Thought (CoT) prompting to more sophisticated structures like Tree of Thoughts (ToT) and Graph of Thoughts (GoT). These methods allow agents to explore multiple reasoning paths, backtrack, and look ahead. However, a significant "comprehension bottleneck" has emerged.
Traditionally, researchers describe the entire graph of previous reasoning attempts in natural language within the prompt. This approach is inherently inefficient, consuming excessive context tokens and often failing to convey the underlying structural dependencies. LLMs frequently struggle to maintain a coherent "mental map" of large, branching reasoning graphs through text alone, leading to errors in state estimation and strategic planning. The Encoder of Thoughts (EoT) framework addresses this by moving beyond textual descriptions to direct structural embeddings.
Methodology: The Thought Encoder
The core of the EoT framework is a plug-and-play module designed to bridge the gap between structured graph data and the LLM’s latent space. Instead of serializing a graph into text, EoT utilizes a Graph Transformer to process the "action graph"—the history of thoughts and actions generated during the reasoning process.
1. Structural Encoding
The Thought Encoder initializes node features using the LLM’s own word embeddings for the text contained within each node. A Graph Transformer then performs message passing to capture the relationships and dependencies between different reasoning steps. Finally, a linear Projector maps the GNN’s output into the exact dimensionality of the LLM’s input space.
2. Integration and Search
These structural embeddings are treated as "special tokens" and prepended to the LLM’s input sequence. This allows the model to perceive the global structure of the search tree or graph without the overhead of a massive text-based summary. EoT is designed to synergize with standard search algorithms, such as Breadth-First Search (EoT-BFS) and Monte Carlo Tree Search (EoT-MCTS), to navigate the reasoning space more effectively.
Analysis: Overcoming the Representation Bottleneck
A key finding of this research is that structural embeddings are fundamentally more efficient than textual descriptions for communicating reasoning state. As the reasoning graph grows in complexity (number of nodes), the perplexity—or uncertainty—of the LLM increases sharply when relying on text-based graph descriptions.
As shown in the perplexity analysis, EoT maintains a consistently low perplexity regardless of graph size, whereas "Nodes only" and "Graph Description" methods show a marked increase in uncertainty. This suggests that EoT provides a clearer, more stable representation of the reasoning state, allowing the agent to make better-informed decisions in complex environments.
Experimental Results
The framework was evaluated across several benchmarks, including GSM8K (math), MATH (advanced math), and Blockworlds (symbolic planning). Notably, the encoder was trained using data automatically generated by a Llama-3 8B model, demonstrating that the system does not require expensive manual annotation.
- Mathematical Reasoning: On the GSM8K dataset using Llama-3 8B, EoT-BFS achieved an accuracy of 88.3%, significantly outperforming vanilla ToT-BFS (80.3%) and prompt-tuning methods (83.0%). On the more challenging MATH dataset, EoT-MCTS reached 37.4% accuracy, compared to 32.1% for standard MCTS.
- Complex Planning: In the Blockworlds environment, which requires long-horizon planning, the gains were even more pronounced. In 12-step planning tasks where standard LLM agents often fail entirely (0% success), EoT-MCTS achieved a 13% success rate.
- Zero-Shot Generalization: The encoder demonstrated strong generalization capabilities; when trained on simpler GSM8K problems, it still provided significant performance boosts on the more complex MATH dataset, suggesting it learns a generalizable method for navigating reasoning structures.
Conclusion
The Encoder of Thoughts framework represents a significant step toward more capable language agents by addressing the structural comprehension limits of current LLMs. By aligning graph-based reasoning structures with the LLM's input space through structural embeddings, EoT reduces token redundancy and improves planning accuracy. Future work will explore the application of EoT to more open-ended tasks like code generation and the optimization of GNN architectures to prevent overfitting in increasingly deep reasoning graphs.
Summary updated