TL;DR
MINDGYM is a novel data synthesis framework that enables large vision-language models (VLMs) to self-generate high-quality, cognitively diverse multi-hop reasoning data. By fine-tuning models on just 400 of these structured, self-generated samples, MINDGYM achieves significant performance boosts across multiple text and multimodal reasoning benchmarks, outperforming baselines trained on up to ten times more data.
What Problem Does This Paper Address?
While large foundation models excel at many tasks, training them to possess robust, transferable reasoning abilities remains a major challenge. Current instruction-tuning approaches typically rely on two paradigms, both of which have bottlenecks:
- Manual curation: Datasets like OK-VQA or ScienceQA are high-quality but labor-intensive and expensive to scale.
- Automated synthesis: Methods like MMEvol or MMInstruct scale easily but often produce shallow, repetitive, or logically inconsistent data.
Furthermore, alternative methods like reinforcement learning (RL) or iterative prompting incur prohibitive computational costs. MINDGYM addresses this gap by introducing a data-centric, self-challenging paradigm: enabling models to systematically target their own cognitive bottlenecks by generating structured, multi-hop questions.
Methods
MINDGYM breaks down question synthesis into a structured four-stage pipeline that injects human-like cognitive traits (breadth, depth, and difficulty progression) directly into the prompt design.
The framework consists of three core components:
- Cognitive Thinking Process Injection: Adapting protocols like Thinking-Claude, the framework prompts the model to externalize its internal stream of consciousness (e.g., hypothesis formulation, counterfactual reasoning, self-verification) before generating data.
- Seed Single-Hop Question Synthesis: The model generates atomic, single-hop question-answer primitives grounded in a self-generated background context across eight meta-topics (e.g., causal inference, economic principles). A reject sampling mechanism filters out semantically redundant questions using cosine similarity.
- Challenging Multi-Hop QA Synthesis: The model uses thinking-centric operators—such as Bridging, Comparison, and Temporal reasoning—to compose the atomic seeds into complex, multi-hop questions with explicit reasoning trajectories.
Finally, instead of uniform training, MINDGYM employs a Structured Progression Training pathway. The model transitions from guided answering (receiving the question and reasoning trace) to reason reconstruction, paired reasoning, and ultimately autonomous solving, mirroring human cognitive development.
Datasets
The authors evaluated MINDGYM across six prominent reasoning benchmarks spanning two modalities:
- Text-Based Benchmarks: GSM8K (grade-school math), MATH (competition-level math), and GPQA (graduate-level Google-proof Q&A).
- Multimodal Benchmarks: MMStar, MathVista-Mini, and MathVision-Mini.
MINDGYM's 400-sample synthetic datasets were compared against established SOTA text and multimodal datasets, including LIMO (817 curated logical chains), Open-O1 (subsampled to 400 and 4,000 samples), and core subsets of MMEvol (ScienceQA and DvQA).
Key Insights and Ideas
- Quality Variance is a Silent Killer: Using the Data-Juicer toolkit, the authors discovered that high average data quality is only half the battle. Reducing quality variance (standard deviation) is absolutely critical for stable fine-tuning. Larger models naturally generate synthetic data with significantly lower variance, making them excellent data synthesizers.
- Structure Trumps Factual Perfection: In an ablation study, removing data flagged as "logically flawed" or "hallucinatory" by GPT-4o actually degraded downstream model performance. This suggests that for thinking-centric tuning, the diversity and structure of the reasoning trajectories matter more than strict factual precision.
- Cross-Modal Transferability: Even when MINDGYM synthesized data in a pure-text format, it significantly boosted performance on vision-intensive benchmarks like MathVision. Strengthening the model's intrinsic, abstract thinking capabilities naturally propagates improvements across modalities without explicit visual supervision.
Key Results
Despite using a tiny fraction of the data volume used by traditional pipelines, MINDGYM achieved remarkable results across different model architectures (Qwen and InternVL series) and scales (7B to 38B parameters):
- Massive Efficiency Gains: With only 400 synthetic samples, MINDGYM boosted the performance of Qwen2.5-VL-7B on MathVision-Mini by over 16%, outperforming Open-O1 baselines trained on 10× more data.
- Superior Data Properties: Data-Juicer analysis showed that MINDGYM's synthetic data achieved a 16.7% higher average quality and a 67.91% reduction in quality variance compared to baseline sources.
- Win-Rate Dominance: Evaluated by GPT-4 as an expert scorer, MINDGYM achieved an overall win-rate improvement of 15.8% in thinking depth and breadth over competitive baselines.
Limitations and Future Work
The authors note that extending MINDGYM to fully multimodal synthesis remains an open challenge. Currently, generating text-image QA pairs relies on static anchor image datasets (like ScienceQA) to prevent the model from hallucinating visual elements.
To achieve true self-evolution, future work will focus on:
- Incorporating richer, more diverse image datasets.
- Developing fully generative pipelines where the VLM synthesizes both the image and the associated reasoning task simultaneously.
- Expanding cognitive relationships into dynamic visual scenarios (video) and specialized domains like medicine and finance.
Summary updated