Introduction
As Large Language Models (LLMs) evolve toward "test-time scaling"—the ability to think longer before providing an answer—the demand for high-quality, multi-step reasoning data has skyrocketed. However, Olympiad-level mathematical problems are notoriously scarce. Existing synthetic data generation methods, such as prompt-based evolution or corpus mining, often result in repetitive, logically shallow, or static datasets.
CogAtom addresses this bottleneck by shifting the focus from linear text generation to structured synthesis. It breaks down human-authored solutions into "cognitive atoms"—the fundamental units of reasoning—and recombines them using a diversity-promoting graph-based algorithm. This approach generates a massive, logically rigorous dataset that significantly enhances the mathematical reasoning capabilities of LLMs, enabling them to tackle elite competition benchmarks like the American Invitational Mathematics Examination (AIME).
The CogAtom Framework
The CogAtom framework operates through a multi-stage process designed to capture the "cognitive associations" human experts use to bridge disparate mathematical concepts.
- Reasoning Atom Extraction: Using a seed set of approximately 9,400 high-quality problems (from sources like GSM8K, MATH, and JEEE), the system extracts "cognitive atoms"—atomic knowledge entities such as "Prime Factorization" or "Modular Arithmetic." This process, powered by GPT-4o, resulted in over 44,000 unique atoms after clustering to remove redundancy.
- Graph-Based Chain Generation: A Cognitive Association Graph is constructed where nodes represent atoms and edges represent their co-occurrence in human solutions. To generate novel problem "skeletons," the framework employs Diversity-Promoting Degree-Regularized Path Expansion (DPDRPE), a random walk algorithm that favors less common conceptual connections.
- Refinement and Synthesis: Refinement operators like Path Extension, Bridge Replacement, and Counterfactual Perturbation ensure logical flow and creative conceptual leaps. These refined chains serve as blueprints for an LLM to generate coherent problem statements and detailed step-by-step solutions.
Experimental Results
The CogAtom-trained models demonstrate substantial improvements over state-of-the-art (SOTA) baselines, including Evol-Instruct and NuminaMath.
As shown in the table above, a Qwen2.5-Math-7B model fine-tuned on CogAtom data (CogAtom-long) achieved 83.0% accuracy on MATH500, significantly outperforming the next best baseline (73.8%). On the elite AIME 2024 benchmark, the framework enabled models to solve significantly more problems than standard synthetic methods. For instance, a CogAtom-long model using a DeepSeek-R1 distilled base solved 10/30 problems, more than doubling the performance of several baselines. Furthermore, the researchers observed a clear monotonic improvement in performance as the synthetic data scale increased from 300k to 1.6M samples, particularly on the most challenging benchmarks.
Key Insights
- Complexity through Combinatorics: Olympiad-level difficulty arises from the interplay of multiple concepts. Modeling this as a graph traversal problem allows for the creation of a near-infinite space of valid, complex reasoning paths.
- Controllable Difficulty: By adjusting the number of cognitive atoms in a generated chain (targeting ~10 for Olympiad level), the framework can precisely tune the reasoning depth of the output.
- Cross-Domain Generalization: The framework's success in improving physics reasoning (evaluated on MMLU Physics) suggests that "cognitive atoms" are a universal way to model complex human thought across STEM fields.
Limitations and Future Work
Currently, CogAtom is limited to textual modalities and cannot generate problems involving visual elements like geometry diagrams. It also relies on a strong "teacher model" (like GPT-4o) for initial atom extraction. Future work aims to extend CogAtom into a multimodal framework, representing visual components as a new type of cognitive atom to synthesize geometry and physics problems requiring spatial reasoning.
Summary updated