Abstract
This paper presents ReasonFormer, a unified reasoning framework designed to mirror the modular and compositional reasoning process of humans in complex decision-making. Inspired by dual-process theory in cognitive science, ReasonFormer decouples the representation module (System 1: automatic thinking) from reasoning modules (System 2: controlled thinking) to capture different levels of cognition. Built upon the representation module, specialized reasoning modules are pre-trained in fundamental skills such as logic and extraction. To mimic controlled compositional thinking, these modules are dynamically activated and composed in both parallel and cascaded manners. Evaluated on 11 datasets, ReasonFormer demonstrates substantial performance boosts and superior generalization in few-shot learning, revealing the power of modular, compositional reasoning.
Introduction: The Need for Modular Reasoning
Prevailing Large Language Models (LLMs) typically employ a "homogeneous" architecture, processing every input through a fixed, monolithic sequence of layers. While effective for many tasks, this approach struggles to explicitly model the complex, modular reasoning processes humans use.
In cognitive science, Dual-Process Theory posits that human cognition consists of two systems: System 1 (fast, intuitive, and automatic) and System 2 (slow, logical, and controlled). Current LLMs essentially collapse these two systems into a single computational "call." This makes it difficult for models to handle tasks requiring specific, cascaded skills—such as recalling a factual detail, performing a logical deduction based on that detail, and then extracting a final answer. The authors of ReasonFormer aim to "disentangle" these capabilities, creating a model that can dynamically decide which skills to use and how "deeply" to process any given problem.
The ReasonFormer Framework
ReasonFormer is a unified framework built on an encoder-decoder (T5) architecture, introducing several novel components to achieve modularity and compositionality.
Decoupled Modules
The encoder is split into two distinct parts. The first nine layers function as the Representation Module (System 1), focusing on contextual understanding and semantic representation. The final layers are replaced by a set of specialized Reasoning Modules (RMs) (System 2).
Specialized Reasoning Modules
The RMs are pre-trained to be "professionals" in five fundamental reasoning skills:
- Logic: For deductive reasoning.
- Simple QA: For answer extraction.
- NER: For named entity recognition.
- NLI: For determining semantic relevance and natural language inference.
- Factual Knowledge: For memorization and retrieval of facts.
Dynamic Composition and Control
For every input, a Skill Router dynamically calculates which RMs to activate at each step. Simultaneously, a Stop Gate decides whether the current reasoning is sufficient or if another "cascaded" step is required. Unlike standard Mixture-of-Experts (MoE) models that activate experts in parallel, ReasonFormer can stack these modules. For example, it might pass information through a "Fact" module in step one and a "Logic" module in step two, allowing for complex, multi-step reasoning chains.
Experimental Setup
The model was evaluated on 11 diverse datasets requiring various reasoning skills:
- Logical Reasoning: ReClor
- Commonsense Knowledge: CSQA, ARC, PIQA, HellaSwag
- Natural Language Inference: aNLI
- Multi-hop QA: HotpotQA, WikiHop
- Dialogue Reasoning: MuTual, DREAM
- General Reading Comprehension: RACE
The specialized RMs were pre-trained using a mix of self-supervised and semi-supervised corpora, including LogicGAN for logic, ProQA for QA, and Wikidata for facts and NER.
Key Insights
Modularity and Generalization
Few-shot experiments revealed that specialized modules could be "frozen" while still allowing the model to adapt to new tasks. By keeping the reasoning modules fixed and only training the router and adapters, the model demonstrated that it had truly learned "skills" that can be repurposed without catastrophic forgetting.
The Importance of Depth
Analysis of the "Stop Gate" showed that different tasks require varying reasoning depths. Simpler tasks often stop early, while complex multi-hop questions consistently utilize the maximum reasoning depth, mirroring human deliberate thinking.
Cascaded vs. Parallel Reasoning
ReasonFormer significantly outperformed a Mixture-of-Reasoning-Modules (MoRM) baseline that only uses parallel experts. This suggests that the ability to sequence different skills is more critical for complex reasoning than simply having a variety of skills available simultaneously.
Key Results
ReasonFormer demonstrated state-of-the-art performance, outperforming both the vanilla T5 base and a "Reasoning Pre-Trained" T5 (RPT-T5) across all 11 datasets.
On average, ReasonFormer achieved a score of 62.1, a significant improvement over the 54.3 of Vanilla T5 and 57.2 of RPT-T5. Furthermore, the model maintained high efficiency; using "Sparse Activation" (top-2 modules per step), ReasonFormer achieved these results while reducing the number of activated parameters compared to full activation.
Conclusion and Future Directions
ReasonFormer validates the cognitive science-inspired approach of decoupling intuitive representation from deliberate reasoning. By modularizing reasoning skills, the framework achieves better performance, efficiency, and few-shot generalization.
Future work may involve expanding the library of reasoning modules to include specialized domains like numerical reasoning or code. Additionally, exploring how this modularity performs in decoder-only architectures (like GPT) could further enhance the scalability and applicability of compositional reasoning in large-scale language models.
Summary updated