Traditional model distillation focuses on teaching a "student" model to mimic the final outputs of a "teacher." While effective for simple classification or sentiment analysis, this approach fails for complex problem-solving. If a student only sees the final answer to a calculus problem, they may memorize the result but will never learn the underlying rules of integration.
Reasoning Trajectory Distillation shifts the focus from the destination to the journey. Instead of training on input-output pairs, researchers are fine-tuning smaller models on the intermediate "trajectories"—the step-by-step chains of thought, tool calls, and self-corrections—generated by larger models or complex multi-agent systems.
From Labels to Logical Paths
The core intuition is that reasoning is a transferable skill that can be internalized into a model's weights. By treating reasoning as a sequence of discrete, logical actions, we can provide a much denser supervision signal than a simple label.
Jianshu Chen and Shiyang Li demonstrated that small language models (SLMs) become significantly more capable when they "learn to explain." By training a 3B-parameter model on the intermediate reasoning steps of GPT-3, they created a student that could outperform its 175B-parameter teacher on commonsense reasoning tasks. This suggests that the bottleneck for small models isn't necessarily their parameter count, but the quality of the logic they are exposed to during training.
Internalizing the Agentic Loop
Modern reasoning often involves more than just "thinking"; it involves "doing." Models must decide when to use a calculator, search the web, or run a snippet of code. Trajectory distillation allows smaller models to inherit these "agentic" behaviors.
Frameworks like Husky, developed by Joongwon Kim and Hannaneh Hajishirzi, use high-quality solution trajectories to teach 7B-parameter models how to coordinate specialized tools. Similarly, the ToRA framework by Zhibin Gou and Yeyun Gong distills structured interaction logs—containing reasoning steps and code execution results—to bridge the gap between linguistic logic and computational precision.
By training on these trajectories, the student model doesn't just learn the answer; it learns the strategy of when to stop thinking and start calculating.
Distilling Collective Intelligence
One of the most powerful applications of this idea is distilling the "wisdom of a crowd" into a single model. Researchers are now using complex, computationally expensive systems—like multi-agent flows or massive search algorithms—to generate "gold standard" trajectories that a single model could never produce on its own.
- Multi-Agent Distillation: Yihe Deng and Paul Mineiro introduced Flow-DPO, which distills the collaborative reasoning of a multi-agent "flow" into a single model. This allows a standalone model to internalize deliberative patterns that previously required multiple agents to coordinate.
- Search-to-Weights: Zhenting Qi and Li Lyna Zhang explore how the sophisticated reasoning found during expensive test-time searches (like Monte Carlo Tree Search) can be distilled back into a base model's parameters, potentially removing the need for heavy search at inference time.
Expanding the Reasoning Frontier
Why is this more effective than just letting a model learn through trial and error? Prasanna Mayilvahanan and Wieland Brendel found that distilling long-form reasoning trajectories is currently more effective at expanding a model's problem-solving frontier than pure reinforcement learning (RL).
In high-entropy search spaces—like complex math or coding—autonomous discovery is slow and difficult. Explicit guidance through complex reasoning steps provides a "warmup" that allows models to acquire sophisticated thinking patterns with a fraction of the data. As Safal Shrestha and Keith W. Ross argue, training on verbose reasoning traces—including backtracking and verification—allows even 1B-parameter models to internalize the process of thinking, leading to massive gains in generalization.
By capturing the "how" instead of just the "what," trajectory distillation transforms small models from simple pattern matchers into disciplined reasoners.