Abstract: Large Language Models (LLMs) have recently demonstrated exceptional performance in various Natural Language Processing (NLP) tasks. They have also shown the ability to perform chain-of-thought (CoT) reasoning to solve complex problems. Recent studies have explored CoT reasoning in complex multimodal scenarios, such as the science question answering task, by fine-tuning multimodal models with high-quality human-annotated CoT rationales. However, collecting high-quality COT rationales is usually time-consuming and costly. Besides, the annotated rationales are hardly accurate due to the external essential information missed. To address these issues, we propose a novel method termed T-SciQ that aims at teaching science question answering with LLM signals. The T-SciQ approach generates high-quality CoT rationales as teaching signals and is advanced to train much smaller models to perform CoT reasoning in complex modalities. Additionally, we introduce a novel data mixing strategy to produce more effective teaching data samples by policy for simple and complex science question answer problems. Extensive experimental results show that our T-SciQ method achieves a new state-of-the-art performance on the ScienceQA benchmark, with an accuracy of 96.18%. Moreover, our approach outperforms the most powerful fine-tuned baseline by 4.5%.
Introduction
Solving scientific problems requires more than simple pattern matching; it necessitates multi-step reasoning and a vast repository of external knowledge. While Chain-of-Thought (CoT) prompting has enabled Large Language Models (LLMs) to excel at these tasks, smaller multimodal models—which are more efficient for deployment—often struggle to replicate this reasoning depth.
Traditionally, these smaller models are trained using human-annotated rationales. However, human labels suffer from two major limitations. First, the cost and scale required to write detailed step-by-step explanations for thousands of scientific problems are prohibitive. Second, human annotators often provide "thin" explanations that assume prior knowledge, missing the explicit external context necessary for a model to learn underlying logic effectively.
Methodology
The researchers propose T-SciQ, a three-stage pipeline designed to distill the reasoning prowess of a "Teacher" LLM (such as GPT-3.5) into a more efficient "Student" model.
- Generating Teaching Data: The framework generates two types of rationales. QA-CoT provides standard step-by-step reasoning guided by the correct answer. For more complex problems, QA-PCoT (Plan-based CoT) uses a "Plan-and-Solve" prompt where the teacher generates a lecture, devises a multi-step plan, and then executes it.
- Mixed Data Strategy: Recognizing that not all problems require complex planning, the authors developed a skill-based mixing strategy. By evaluating which skills (e.g., "Reading a map" vs. "Identifying isotopes") benefit most from standard vs. plan-based CoT, they created an optimized training set.
- Two-Stage Fine-Tuning: The student model, SciStudent, is trained first to generate a rationale based on the question and image, and subsequently to use that self-generated rationale to infer the final answer.
Analysis and Insights
A core finding of this research is that LLMs can be superior teachers compared to human annotators. Human rationales are often brief, whereas LLM-generated rationales provide essential external context that helps the student model learn broader relationships.
Furthermore, the "Plan-and-Solve" approach prevents "reasoning drift" in complex tasks. By forcing the model to generate a plan before the rationale, T-SciQ keeps the logic grounded and structured. The skill-based mixing ensures that simple problems are not over-complicated while hard problems receive the necessary depth of reasoning.
Experimental Results
The T-SciQ method was evaluated on the ScienceQA benchmark, a massive dataset of 21,208 multimodal questions across 379 skills. Key results include:
- New State-of-the-Art: T-SciQ achieved a record 96.18% accuracy on ScienceQA.
- Superiority over Baselines: This performance is 4.5% higher than the previous best fine-tuned model and outperforms GPT-4's few-shot performance by 9.64%.
- Surpassing Human Performance: The model exceeded average human performance (88.40%) by nearly 8%.
- Efficiency: These gains were achieved using student models that are significantly smaller and more computationally efficient than the teacher LLMs used for data generation.
Conclusion
T-SciQ demonstrates that high-quality LLM-generated signals can effectively teach complex reasoning to smaller multimodal models. While the current framework relies on powerful closed-source teachers like GPT-3.5, it sets a new benchmark for science question answering. Future work may explore diverse LLM teachers and parameter-efficient fine-tuning (PEFT) to further optimize student model performance and investigate methods to mitigate potential teacher hallucinations.
Summary updated