Abstract
While Large Language Models (LLMs) have demonstrated remarkable reasoning improvements through Chain of Thought (CoT) techniques, research has largely focused on technical implementation rather than alignment with human cognitive processes. This paper provides a comprehensive survey of CoT fine-tuning by adopting Edward de Bono’s "Six Thinking Hats" framework. By categorizing AI reasoning into six human-like abilities—planning, divergent thinking, intuition, reflection, internal thinking, and fact perception—this work offers a roadmap for evolving LLMs from simple "thinkers" to sophisticated "insight models."
The Evolution of Reasoning Models
The paper defines a three-stage evolution of AI reasoning capability, moving from superficial responses to complex, multi-faceted cognition:
- Reflex Model: Characterized by immediate and superficial responses without deep processing.
- Thinking Model: Employs pre-thinking and one-way reasoning paths to solve problems step-by-step.
- Insight Model: The current frontier, where models "put on thinking hats" to engage in interactive, strategic, and human-like reasoning.
A Taxonomy of Reasoning: The Six Thinking Hats
To bridge the gap between machine learning algorithms and cognitive psychology, the authors introduce a bi-level taxonomy based on the Six Thinking Hats framework. This allows for a systematic analysis of advanced capabilities like self-correction and strategic planning.
- Blue Hat (Management/Planning): Focuses on global and local reasoning roadmaps (e.g.,
CodePlan,LUMOS). - Green Hat (Creativity/Divergent Thinking): Explores multiple solution paths through methods like Mixture-of-Experts and test-time diversity.
- Red Hat (Intuition/Preference): Aligns models with human-preferred reasoning styles, such as conciseness or safety (e.g.,
Logic-RL). - Black Hat (Criticism/Reflection): Enables models to question, debate, and refine their own logic (e.g., Multi-agent debate, self-reflection).
- Yellow Hat (Optimism/Internal Thinking): Accelerates reasoning by internalizing steps or "skipping" unimportant chunks (e.g.,
Skip-thinking, latent reasoning). - White Hat (Objectivity/Fact Perception): Integrates external tools and Retrieval-Augmented Generation (RAG) to ground reasoning in objective facts (e.g.,
ToolkenGPT).
Technical Pathways: SFT and RFT
The survey identifies two primary technical pathways for achieving these reasoning capabilities:
Supervised Fine-Tuning (SFT)
SFT focuses on rationale annotation—either manual or automatic—and training methods such as pre-thinking (reasoning before the answer), post-thinking, and multi-task learning. These methods are essential for establishing the initial reasoning patterns in "Thinking Models."
Reinforced Fine-Tuning (RFT)
RFT represents the shift toward more autonomous reasoning. While early methods relied on complex PPO-like setups requiring multiple models, newer approaches like GRPO (Group Relative Policy Optimization) and DPO (Direct Preference Optimization) significantly reduce computational overhead while improving performance in complex domains like math and coding.
Key Insights and Future Directions
- The "Clash of Caps": Different reasoning modes can conflict. For instance, rigorous planning (Blue Hat) may be slowed by constant reflection (Black Hat). Finding the optimal coordination of these "hats" is a critical research frontier.
- Internalized Reasoning: As models mature, they can move from explicit CoT (writing every step) to latent reasoning (thinking "in their head"), enhancing efficiency without losing depth.
- Meta-Planning: Current models excel at planning for specific tasks but lack a general, task-independent strategy for how to plan.
- Methodological Diversity: Most current "diverse thinking" is limited to surface-level wording changes. Future research must enable models to employ distinct logical methods (e.g., inductive vs. deductive) for the same problem.
Conclusion
By framing CoT fine-tuning through the lens of human reasoning mechanisms, this survey provides a structured path toward more robust and efficient AI. The transition from "Thinking Models" to "Insight Models" requires not just more data, but a sophisticated coordination of cognitive-like abilities that mirror the complexity of human thought.
Summary updated