Abstract
Large language models (LLMs) typically operate as "black boxes," processing inputs through a fixed stack of layers in a single pass. This monolithic approach lacks the modularity of human cognition, which often separates intuitive understanding from deliberate, multi-step reasoning. In this paper, presented at ACL 2023, we introduce ReasonFormer, a unified framework that disentangles specific reasoning skills—such as logic, fact retrieval, and natural language inference—into modular components. By dynamically composing these modules in parallel or sequence, ReasonFormer improves both performance and interpretability across complex tasks, outperforming standard models and demonstrating robust generalization.
Introduction: The Limitations of Monolithic Models
Prevailing language models are largely "homogeneous." They apply the same computational path to every prompt, regardless of complexity. This architecture leads to three primary challenges:
- Interpretability: The internal reasoning process remains opaque, making it difficult to trace how a model reached a specific conclusion.
- Generalization: Learning new tasks often requires retraining the entire model rather than recombining existing skills.
- Efficiency: Models often expend the same amount of "thinking" on simple questions as they do on complex ones.
ReasonFormer addresses these issues by adopting a modular architecture inspired by cognitive science’s Dual-Process Theory, which distinguishes between fast, intuitive "System 1" thinking and slow, deliberate "System 2" reasoning.
Methodology: The ReasonFormer Framework
ReasonFormer utilizes an encoder-decoder architecture (based on T5) where the encoder is reimagined to support modular skill composition.
System 1: Representation Module
The initial layers of the encoder focus on "automatic thinking," generating contextual representations of the input. This serves as the intuitive foundation for subsequent reasoning.
System 2: Compositional Reasoning Modules
The core of the framework consists of modular "experts" pre-trained on five fundamental skills: Logic, Simple QA, Named Entity Recognition (NER), Natural Language Inference (NLI), and Factual Knowledge.
Key components of the System 2 process include:
- Reasoning Router & Stop Gate: At each reasoning step, the Router decides which skills to activate (parallel composition). The Stop Gate then determines if the reasoning is sufficient or if another step is required (cascaded composition).
- Reasoning Adapters: Lightweight adapters tune shared reasoning modules for different depths, allowing a skill like "Logic" to be applied differently at various stages of the thought process.
Experimental Setup
The model was evaluated on 11 diverse datasets requiring various types of reasoning, including logic (ReClor), commonsense (CSQA, ARC, PIQA, HellaSwag), and multi-hop QA (WikiHop, HotpotQA). Training occurred in two stages: specialized skill pre-training on massive specialized corpora, followed by downstream adaptation to learn routing and composition.
Key Results and Discussion
ReasonFormer achieved state-of-the-art performance, reaching an average accuracy of 62.1% across all 11 tasks. This represents a significant improvement over the vanilla T5 baseline (54.3%) and the Mixture-of-Reasoning (MoRM) baseline (59.2%).
Insights from the Modular Approach
- Cascading vs. Parallelism: Multi-step "cascaded" reasoning significantly outperforms parallel-only reasoning. Feeding the output of one reasoning step into the next allows for deeper problem-solving.
- Skill Modularity: Case studies confirm that the Router activates appropriate skills for specific tasks—for instance, leaning on "Fact" and "QA" modules for commonsense tasks, and "NER" and "NLI" for inference tasks.
- Efficiency via Sparsity: The "Sparse" version of ReasonFormer (activating only the top-2 skills per step) utilized 294M parameters compared to the 407M of the "Full" version, yet it maintained competitive performance (61.8% vs 62.1%).
Conclusion
ReasonFormer demonstrates that decoupling reasoning from representation allows for more interpretable and generalizable language models. By mimicking the human dual-process system, the framework enables LLMs to deliberately chain specialized skills to solve complex problems. Future work will explore the self-supervised discovery of new reasoning modules and the integration of numerical reasoning capabilities.
Summary updated