Learn to Think: Bootstrapping LLM Logic Through Graph Representation Learning

Hang Gao, Chenhao Zhang, Tie Wang, Junsuo Zhao, Fengge Wu, Changwen Zheng, Huaping Liu

Introduction

While Large Language Models (LLMs) have demonstrated significant reasoning potential through prompting techniques like Chain-of-Thought (CoT) and Tree-of-Thought (ToT), these approaches often encounter performance plateaus due to their inherent rigidity and heavy reliance on task-specific prompt engineering. Current paradigms typically follow predefined, static reasoning paths that fail to adapt to the evolving complexity of a problem as it unfolds.

The "Learn to Think" (L2T) framework addresses these bottlenecks by modeling the reasoning process as a dynamic graph. By integrating Graph Neural Networks (GNNs) and reinforcement learning, L2T enables LLMs to bootstrap their own logic. This allows for flexible navigation of problem spaces—including backtracking and strategy shifts—without requiring meticulously hand-crafted prompts for every new task.

The L2T Framework

The L2T architecture moves beyond static text generation by treating reasoning as a structured search problem guided by a learned policy.

The L2T framework architecture, illustrating the iterative process of node classification, GNN-based mode selection, and thought generation across multiple steps.

1. Reasoning Process Graph (RPG)

At the heart of L2T is the Reasoning Process Graph $G = \{V, E\}$, where each node represents an intermediate "thought" or reasoning step. Unlike linear chains, this graph structure allows the model to explore multiple branches and revisit previous states. Nodes are dynamically classified into categories such as Continue, Stop, Backtrack, or Final Result.

An example of a Reasoning Process Graph for the Game of 24, showing how nodes are processed and classified to reach a final result.

2. GNN-based Mode Selection

To navigate this graph efficiently, L2T employs a lightweight Graph Neural Network (GNN). This module analyzes the current topology of the RPG to predict the optimal "reasoning mode." Instead of generating text directly, the GNN adjusts critical hyperparameters—such as temperature, top-p, and branching factors—allowing the LLM to adapt its creative or precise "thinking style" based on the current state of the problem.

3. Reinforcement Learning Optimization

The GNN is trained using Proximal Policy Optimization (PPO). The system is rewarded for reaching correct solutions and receives intermediate "shaping" rewards for logical progress. This training allows the model to internalize effective search strategies across diverse domains, effectively learning how to think rather than just what to output.

Experimental Evaluation

The authors benchmarked L2T against state-of-the-art methods (CoT, ToT, GoT, and AoT) using GPT-4o as the backbone. The evaluation spanned four distinct task types:

  • Sudoku (3x3 to 5x5): Logic and constraint satisfaction.
  • Game of 24: Mathematical search and arithmetic.
  • TruthQuest: Logical verification and suppositional reasoning.
  • Creative Writing: Open-ended conceptual generation.

A key component of the evaluation was the "w/o TSP" (without Task-Specific Prompts) setting, which tested the framework's ability to generalize without manual guidance.

Results and Key Insights

L2T demonstrated superior performance and efficiency across all benchmarks:

  • Accuracy: In 4x4 Sudoku, L2T achieved 98.46% accuracy, significantly outperforming ToT's 72.31%. In the Game of 24, it reached 80.42% accuracy, surpassing the next best method (AoT) at 74.23%.
  • Token Efficiency: L2T is significantly more economical. For the Game of 24, it used an average of 4.68k tokens per case, compared to 11.60k for ToT and 7.56k for GoT.
  • Robustness to Prompting: While traditional methods saw performance drops of up to 50% when task-specific prompts were removed, L2T remained stable. Its logic is embedded in the GNN's policy rather than the text prompt.
  • Adaptive Strategies: Analysis showed the GNN naturally learned task-appropriate behaviors, such as increasing diversity (higher temperature/top-p) for creative writing while maintaining precision for mathematical tasks.

Conclusion

The L2T framework represents a shift from static "prompting" to dynamic "learning" in LLM reasoning. By leveraging graph representation learning, models can navigate complex logic spaces with greater flexibility and efficiency. While the GNN requires an initial training phase, the resulting system is more robust, token-efficient, and capable of solving complex problems without the need for rigid, hand-crafted instructions.

Summary updated

Abstract

Large Language Models (LLMs) have achieved remarkable success across various domains. However, they still face significant challenges, including high computational costs for training and limitations in solving complex reasoning problems. Although existing methods have extended the reasoning capabilities of LLMs through structured paradigms, these approaches often rely on task-specific prompts and predefined reasoning processes, which constrain their flexibility and generalizability. To address these limitations, we propose a novel framework that leverages graph learning to enable more flexible and adaptive reasoning capabilities for LLMs. Specifically, this approach models the reasoning process of a problem as a graph and employs LLM-based graph learning to guide the adaptive generation of each reasoning step. To further enhance the adaptability of the model, we introduce a Graph Neural Network (GNN) module to perform representation learning on the generated reasoning process, enabling real-time adjustments to both the model and the prompt. Experimental results demonstrate that this method significantly improves reasoning performance across multiple tasks without requiring additional training or task-specific prompt design. Code can be found in https://github.com/zch65458525/L2T.
BibTeX
@inproceedings{gao2025learn,
  author = {Hang Gao and Chenhao Zhang and Tie Wang and Junsuo Zhao and Fengge Wu and Changwen Zheng and Huaping Liu},
  title = {{Learn to Think: Bootstrapping LLM Logic Through Graph Representation Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/896},
}
Figure