Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism

Xiaoshu Chen, Sihang Zhou, Ke Liang, Duanyang Yuan, Haoyuan Chen, Xiaoyu Sun, Linyuan Meng, Xinwang LiuarXiv 2025 · DOI · arXiv

Abstract

While Large Language Models (LLMs) have demonstrated remarkable reasoning improvements through Chain of Thought (CoT) techniques, research has largely focused on technical implementation rather than alignment with human cognitive processes. This paper provides a comprehensive survey of CoT fine-tuning by adopting Edward de Bono’s "Six Thinking Hats" framework. By categorizing AI reasoning into six human-like abilities—planning, divergent thinking, intuition, reflection, internal thinking, and fact perception—this work offers a roadmap for evolving LLMs from simple "thinkers" to sophisticated "insight models."

The Evolution of Reasoning Models

The paper defines a three-stage evolution of AI reasoning capability, moving from superficial responses to complex, multi-faceted cognition:

  1. Reflex Model: Characterized by immediate and superficial responses without deep processing.
  2. Thinking Model: Employs pre-thinking and one-way reasoning paths to solve problems step-by-step.
  3. Insight Model: The current frontier, where models "put on thinking hats" to engage in interactive, strategic, and human-like reasoning.
The evolution of AI reasoning from the Reflex Model to the Insight Model.

A Taxonomy of Reasoning: The Six Thinking Hats

To bridge the gap between machine learning algorithms and cognitive psychology, the authors introduce a bi-level taxonomy based on the Six Thinking Hats framework. This allows for a systematic analysis of advanced capabilities like self-correction and strategic planning.

A comprehensive taxonomy of CoT fine-tuning methods categorized by the Six Thinking Hats framework.
  • Blue Hat (Management/Planning): Focuses on global and local reasoning roadmaps (e.g., CodePlan, LUMOS).
  • Green Hat (Creativity/Divergent Thinking): Explores multiple solution paths through methods like Mixture-of-Experts and test-time diversity.
  • Red Hat (Intuition/Preference): Aligns models with human-preferred reasoning styles, such as conciseness or safety (e.g., Logic-RL).
  • Black Hat (Criticism/Reflection): Enables models to question, debate, and refine their own logic (e.g., Multi-agent debate, self-reflection).
  • Yellow Hat (Optimism/Internal Thinking): Accelerates reasoning by internalizing steps or "skipping" unimportant chunks (e.g., Skip-thinking, latent reasoning).
  • White Hat (Objectivity/Fact Perception): Integrates external tools and Retrieval-Augmented Generation (RAG) to ground reasoning in objective facts (e.g., ToolkenGPT).

Technical Pathways: SFT and RFT

The survey identifies two primary technical pathways for achieving these reasoning capabilities:

Supervised Fine-Tuning (SFT)

SFT focuses on rationale annotation—either manual or automatic—and training methods such as pre-thinking (reasoning before the answer), post-thinking, and multi-task learning. These methods are essential for establishing the initial reasoning patterns in "Thinking Models."

Reinforced Fine-Tuning (RFT)

RFT represents the shift toward more autonomous reasoning. While early methods relied on complex PPO-like setups requiring multiple models, newer approaches like GRPO (Group Relative Policy Optimization) and DPO (Direct Preference Optimization) significantly reduce computational overhead while improving performance in complex domains like math and coding.

Key Insights and Future Directions

  • The "Clash of Caps": Different reasoning modes can conflict. For instance, rigorous planning (Blue Hat) may be slowed by constant reflection (Black Hat). Finding the optimal coordination of these "hats" is a critical research frontier.
  • Internalized Reasoning: As models mature, they can move from explicit CoT (writing every step) to latent reasoning (thinking "in their head"), enhancing efficiency without losing depth.
  • Meta-Planning: Current models excel at planning for specific tasks but lack a general, task-independent strategy for how to plan.
  • Methodological Diversity: Most current "diverse thinking" is limited to surface-level wording changes. Future research must enable models to employ distinct logical methods (e.g., inductive vs. deductive) for the same problem.

Conclusion

By framing CoT fine-tuning through the lens of human reasoning mechanisms, this survey provides a structured path toward more robust and efficient AI. The transition from "Thinking Models" to "Insight Models" requires not just more data, but a sophisticated coordination of cognitive-like abilities that mirror the complexity of human thought.

Summary updated

Abstract

Chain of thought (CoT) fine-tuning aims to endow large language models (LLMs) with reasoning capabilities by training them on curated reasoning traces. It leverages both supervised and reinforced fine-tuning to cultivate human-like reasoning skills in LLMs, including detailed planning, divergent thinking, intuitive judgment, timely reflection, internal thinking, and fact perception, etc. As CoT fine-tuning has advanced, LLMs have demonstrated substantial improvements in tasks such as mathematical reasoning and code generation. However, existing surveys about CoT fine-tuning primarily focus on technical aspects and overlook a systematic analysis from the perspective of human reasoning mechanisms. Given that the ultimate goal of CoT fine-tuning is to enable LLMs to reason like humans, it is crucial to investigate this technique through the lens of human cognition. To fill this gap, we present the first comprehensive survey of CoT fine-tuning grounded in human reasoning theory. Specifically, inspired by the well-known Six Thinking Hats framework, which systematically characterizes common human thinking modes using six metaphorical hats, we classify and examine CoT fine-tuning methods through this lens. Furthermore, building upon this theory, we outline potential directions for future research in CoT fine-tuning. In addition, we compile a comprehensive overview of existing datasets and model performances, and a real-time GitHub repository \footnote{https://github.com/AI-Chen/Awesome-CoT-Finetuning} that continuously tracks recent advances in this area is maintained. We hope this survey will serve as a valuable resource to inspire innovation and foster progress in this rapidly evolving field.
BibTeX
@misc{chen2025putting,
  author = {Xiaoshu Chen and Sihang Zhou and Ke Liang and Duanyang Yuan and Haoyuan Chen and Xiaoyu Sun and Linyuan Meng and Xinwang Liu},
  title = {{Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism}},
  year = {2025},
  eprint = {2510.13170},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.13170},
}
Figure