Abstract
This work addresses the challenge of democratizing advanced Large Language Models (LLMs) by compressing their mathematical reasoning capabilities into sub-billion parameter Small Language Models (SLMs) without compromising performance. We introduce Equation-of-Thought Distillation (EoTD), a novel technique that encapsulates the reasoning process into equation-based representations to construct an EoTD dataset for fine-tuning SLMs. Additionally, we propose the Ensemble Thoughts Distillation (ETD) framework to enhance the reasoning performance of SLMs. This involves creating a reasoning dataset with multiple thought processes, including Chain-of-Thought (CoT), Program-of-Thought (PoT), and Equation-of-Thought (EoT), and using it for fine-tuning. Our experimental performance demonstrates that EoTD significantly boosts the reasoning abilities of SLMs, while ETD enables these models to achieve state-of-the-art reasoning performance.
Introduction
While Large Language Models (LLMs) like GPT-4 exhibit remarkable mathematical reasoning, their massive computational requirements hinder local deployment and accessibility. Small Language Models (SLMs) offer a path toward democratization but frequently struggle with complex multi-step problems. These failures typically stem from two sources: arithmetic calculation errors and logic gaps in maintaining a coherent reasoning chain. While techniques like Program-of-Thought (PoT) offload calculations to code interpreters, they introduce new failure modes, such as variable definition errors.
Methodology: Equation-of-Thought (EoT)
To bridge the gap between unstructured text and rigid code, we propose Equation-of-Thought (EoT). In this framework, the model is trained to translate natural language problems into systems of linear equations. Unlike Chain-of-Thought (CoT), which relies on the model's internal (and often flawed) arithmetic, or PoT, which requires strict programming syntax, EoT provides a structured mathematical representation that can be solved by a deterministic external solver.
As shown in Figure 1, EoT avoids the "Unknown Variable" pitfalls of PoT by allowing variables to be defined implicitly within a system of equations. By delegating the final computation to an equation solver, we eliminate the calculation errors common in standard CoT.
Ensemble Thoughts Distillation (ETD)
Recognizing that different mathematical problems may favor different reasoning styles, we developed the Ensemble Thoughts Distillation (ETD) framework. This approach creates a diverse training dataset by distilling multiple reasoning paths from a teacher LLM (e.g., GPT-3.5).
The ETD dataset includes:
- CoT Dataset: Natural language step-by-step rationales.
- PoT Dataset: Python-based programmatic solutions.
- EoT Dataset: Systems of linear equations.
During inference, the model employs a hierarchical strategy: it first attempts a PoT solution; if that fails (e.g., code error), it attempts an EoT solution; finally, it falls back to a CoT rationale. This multi-layered approach provides a robust safety net for the SLM.
Experimental Results
We evaluated our approach using the CodeT5 family of models on benchmarks such as GSM8K, ASDiv, and SVAMP. The results were significant:
- Performance Gains: For a 60M parameter model, EoTD improved accuracy on GSM8K from 1.1% to 18.87%. The full ETD ensemble further increased this to 33.58%.
- Punching Above Their Weight: Our CodeT5-Large (770M) model using ETD achieved 42.45% accuracy on GSM8K, outperforming much larger 7B parameter models like Llama-2 (13.3%) and Platypus-2 (14.4%).
Conclusion
Equation-of-Thought Distillation effectively transforms SLMs from "calculators" into "translators," allowing them to leverage external tools for precise arithmetic. By combining this with an ensemble of reasoning styles, we can achieve high-performance mathematical reasoning in models small enough to run on consumer-grade hardware, marking a significant step toward the democratization of AI.
Summary updated