When we want a smaller, more efficient language model to perform at the level of a massive frontier model, our traditional recourse is knowledge distillation or fine-tuning. Both approaches attempt to bake the teacher's capabilities directly into the student's neural weights. However, this parameter-bound approach is computationally expensive, rigid across architectures, and prone to catastrophic forgetting.
A rapidly growing body of research proposes a powerful alternative: decoupling a model's reasoning engine from its experience store. Instead of altering internal parameters, these frameworks externalize an agent's insights, trajectories, and strategies into an explicit, textual memory bank. Because natural language and structured execution logs serve as a universal, model-agnostic medium, a memory bank optimized by one model can be directly plugged into an entirely different architecture at inference time.
A smaller model does not necessarily lack the capacity to execute a complex plan; it simply lacks the capacity to discover or synthesize that plan from scratch. By treating memory as a portable, externalized asset, smaller models can inherit advanced problem-solving behaviors without a single weight update.
Language as a Model-Agnostic API
Why can an external memory bank be shared across entirely different model scales and architectures? The core intuition is that explicit textual memory exists independently of internal parameters.
As demonstrated by Qiushi Sun et al. in Explicit Memory Learning with Expectation Maximization, text-based memory buffers bypass the architectural lock-in of model weights. This allows a smaller language model to significantly enhance its reasoning capabilities simply by reading and utilizing memory states generated by larger, more capable models.
This portability stems from the fact that historical trajectories act as a universal knowledge base. Yunxiang Zhang et al. show in AdaMEM: Test-Time Adaptive Memory for Language Agents that raw experience logs function as a model-agnostic medium. Because the memory is structured as text or sequential actions, diverse agentic systems can parse and execute these strategies regardless of their underlying parameter configurations.
[High-Capacity Teacher] ---> Generates Trajectories/Logs ---> [Shared Memory Bank]
|
v
[Low-Capacity Student] <--- Retrieves Strategies/Context <----------+
By separating the "thinking engine" from the "experience store," agents can evolve dynamically. For instance, Shengtao Zhang et al. introduce a framework called MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory, which isolates a frozen language model from a plastic, updateable episodic memory bank. Because the learned utility scores and trajectories are externalized, a target student model can seamlessly adopt a high-utility memory bank originally generated and validated by a separate teacher model.
Scaling Down Capabilities via In-Context Learning
In practice, this cross-model transfer typically operates by using a high-capacity teacher model to populate a contextual memory that a cheaper student model queries during inference. This allows the student to emulate the reasoning and tool-use patterns of more advanced systems.
In the domain of autonomous agents, Thong Jing Yuan et al. developed RAP: Retrieval-Augmented Planning with Contextual Memory for Multimodal LLM Agents. RAP stores successful execution logs from a teacher model in a shared memory, proving that a robust library of procedural successes can compensate for lower raw reasoning capabilities in smaller local models.
This mechanism serves as a highly affordable deployment strategy. In EcoAssistant: Using LLM Assistant More Affordably and Accurately, Jieyu Zhang et al. use successful execution traces from high-capacity models as in-context demonstrations for lower-capacity models. By querying the cheaper model first and equipping it with the teacher's historical traces, they achieve massive cost reductions while maintaining or exceeding frontier-model accuracy.
Similarly, Sameer Singh et al. explore this in Leveraging In-Context Learning for Language Model Agents. The authors demonstrate that automatically annotating solution trajectories from a large model and strategically "sprinkling" these snippets into the prompt of a smaller model serves as a training-free knowledge transfer mechanism, effectively closing the performance gap between model scales.
Structuring the Shared Knowledge Base
Cross-model memory transfer is rarely a matter of dumping raw text logs into a prompt. To make externalized knowledge truly actionable for a smaller model, the memory must be structured into specialized cognitive abstractions.
Hierarchical Skill Libraries
Instead of forcing a student model to parse long, unstructured histories, the teacher's experiences can be compiled into modular capabilities. Chenxi Wang et al. introduce this in AutoSkills: Automatically Constructing Skill Knowledge Bases for Agents. By structuring agent trajectories into planning, functional, and atomic levels, their framework allows weaker models to inherit a plug-and-play skill library, boosting task success rates by helping the model bypass its native training boundaries.
Error Notebooks
Learning what not to do is often as valuable as learning what to do. In Error Notebook-Guided, Training-Free Part Retrieval in 3D CAD Assemblies via Vision-Language Models, Yunqing Liu et al. have a frontier model reflect on mistakes to build an external "Error Notebook." When smaller models reference these corrected reasoning paths via Retrieval-Augmented Generation (RAG), they inherit complex self-correction capabilities without explicit fine-tuning.
Conceptual Analogies
Sometimes, raw logical steps are too dense for a smaller model to digest. In these cases, high-level abstractions work better. Siyu Yuan et al. explore this in Boosting Scientific Concepts Understanding: Can Analogy from Teacher Models Empower Student Models?. The authors find that using a teacher model to distill complex scientific concepts into free-form analogies acts as a form of "soft" knowledge distillation. These conceptual bridges allow weaker models to solve specialized tasks without needing the full internal logical depth of the teacher.
Optimization Tactics
In data-scarce, highly technical domains like hardware code synthesis, Yujie Zheng et al. implement a value-driven memory approach in Towards Cold-Start Drafting and Continual Refining: A Value-Driven Memory Approach with Application to NPU Kernel Synthesis. By transferring memory banks containing successful execution traces and optimization tactics from high-capability models, they demonstrate that structured external knowledge can directly compensate for weak internal reasoning.
Bidirectional Transfer and Self-Evolution
Because explicit memory is fundamentally model-agnostic, the flow of knowledge does not have to be a one-way street from large to small models. It can also operate bidirectionally or serve as a mechanism for standalone self-evolution.
In a striking reversal of the traditional teacher-student paradigm, Yishuo Cai et al. develop From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memory. Their framework, MemoPilot, demonstrates that strategic insights captured in a learned memory module trained to assist a smaller language model can effectively generalize upward to enhance the performance of a significantly larger, frozen model without requiring additional training.
[Small Model + RL] ---> Optimizes Memory Module ---> [Agile Memory State]
|
v
[Large Frozen Model] <--- Inherits Strategic Insights <-----+
Furthermore, this model-agnostic abstraction allows agents to induce highly transferable operational rules. Po-Chun Chen et al. present Induct-Learn: Short Phrase Prompting with Instruction Induction, a framework where models induce structured instructions and step-by-step reasoning examples from a handful of demonstrations. Because these induced instructions are stored explicitly rather than implicitly in weights, they are highly transferable across scales, allowing student models to achieve substantial performance gains over standard few-shot techniques.
Ultimately, by treating memory as an independent, text-based layer, the AI research community has uncovered a highly efficient path to capability transfer. Language agents no longer need to be massive to act wisely; they simply need access to a well-curated history of wisdom.
Key Foundational Work
Boosting Scientific Concepts Understanding: Can Analogy from Teacher Models Empower Student Models? (2024) — contributed 3 concept elements to this direction.
Explicit Memory Learning with Expectation Maximization (2024) — contributed 2 concept elements to this direction.
RAP: Retrieval-Augmented Planning with Contextual Memory for Multimodal LLM Agents (2024) — contributed 2 concept elements to this direction.