Introduction
Large Language Models (LLMs) have demonstrated remarkable capabilities in solving complex tasks, yet multi-step mathematical reasoning remains a significant challenge. While "Chain-of-Thought" (CoT) prompting—providing step-by-step examples—has emerged as a powerful technique to improve performance, it is constrained by the fixed context windows of modern models. Researchers have developed CoT-Influx, a plug-and-play module designed to maximize the density of effective information within these limits. By packing more high-quality, concise few-shot examples into the prompt, CoT-Influx allows models like LLaMA2-70B to outperform significantly larger counterparts, such as PaLM and Minerva 540B.
The Context Density Problem
The effectiveness of CoT prompting is often hindered by a "Goldilocks" problem involving three primary factors:
- Context Limits: LLMs have a strict token budget. Adding more examples eventually hits a limit that can truncate the model's own reasoning steps, leading to incomplete or incorrect answers.
- Quality vs. Quantity: Performance does not scale linearly with the number of examples. Randomly selected examples can introduce noise, and standard prompt compression tools often remove vital mathematical tokens like numbers or structural operators.
- Model Sensitivity: Different models respond differently to prompt complexity. A smaller model may require simpler logic, while a larger model might only see gains when presented with highly complex reasoning patterns.
CoT-Influx: Reinforced Context Pruning
To address these issues, the authors introduce CoT-Influx, which treats the target LLM as a black box and employs a two-stage pruning process to optimize the prompt context.
Two-Stage Pruning Mechanism
- Stage 1 (Shot Pruner): This stage selects the specific "shots" (examples) from a large candidate pool that are most relevant to the current question. This ensures the model receives the most helpful reasoning patterns.
- Stage 2 (Token Pruner): Once examples are selected, this stage prunes redundant words (e.g., filler text or irrelevant background) while strictly preserving numerical and structural tokens. This "fine-grained" pruning frees up space to include even more examples within the same token budget.
Reinforced Training
Because the pruning process occurs outside the LLM's internal architecture, gradients cannot flow directly. The authors utilize Reinforcement Learning (REINFORCE) to train the pruner. The policy network is optimized using a multi-objective reward function that balances prediction loss, answer correctness ($R_{Acc}$), and a penalty for exceeding token limits. This lightweight architecture uses a frozen BERT-Large model for embeddings and a tiny MLP-based policy network, adding only a ~1.7% increase in latency.
Datasets and Evaluation
The study introduces the Math Reasoning Dataset with Diverse Difficulty (MRD³), a collection of 9.7K pairs. Using GPT-4, the authors "evolved" standard problems from benchmarks like GSM8K and MAWPS into versions with varying difficulty levels. Evaluation was conducted across several rigorous benchmarks, including GSM8K, AddSub, Svamp, and the highly challenging MATH dataset.
Key Results and Insights
The core insight of the paper is that "fewer [tokens] is more [examples]." By pruning internal tokens, the system increases "shot density," providing the LLM with a broader range of reasoning patterns to mimic.
As shown in the results, LLaMA2-70B equipped with CoT-Influx achieved an Exact Match (EM) score of 59.6% on GSM8K. This performance surpasses much larger models, including Minerva (540B) and PaLM (540B), without requiring any fine-tuning of the base LLM. Across six math datasets, CoT-Influx provided up to a 4.40% absolute improvement over standard few-shot baselines.
Furthermore, the system effectively mitigates the "Squeezed Response" effect. The researchers observed that even before hitting a hard token limit, excessively long prompts can crowd the model's internal attention, leading to shorter, lower-quality responses. CoT-Influx prevents this crowding, ensuring the model has the "breathing room" to generate complete reasoning chains.
Conclusion
CoT-Influx demonstrates that the bottleneck in LLM reasoning is often the efficiency of the prompt rather than the model's inherent capacity. While the system currently faces limitations regarding the 512-token window of the BERT embedding model and a dependency on the quality of the initial candidate pool, it represents a significant step toward universal context optimization. Future work may expand this reinforced pruning approach to other domains such as code generation and commonsense reasoning.
Summary updated