Can Test-Time Memory Evolution Outperform Test-Time Training for Scientific Discovery?

Yepeng Liu, Shengmai Chen, Xuandong Zhao, Chengzhi Liu, Dawn Song, Xin Eric Wang, Yuheng BuarXiv 2026 · arXiv

TL;DR

Test-Time Memory Evolution (TTME) is a new, "plug-and-play" method that allows Large Language Models (LLMs) to adapt during scientific discovery by evolving an external memory instead of updating their internal weights. This approach not only outperforms traditional Test-Time Training (TTT) across complex mathematical tasks but does so while slashing computational costs by over 80% and nearly doubling execution speed.

What Problem Does This Paper Address?

In the quest for automated scientific discovery, LLMs are often used in iterative loops: they propose a solution (like a piece of code), evaluate it, and use the feedback to try again. To help the model "learn" from its mistakes during this process, researchers have relied on Test-Time Training (TTT). TTT uses Reinforcement Learning to update the model's weights on the fly.

However, TTT has a major drawback: it is computationally expensive and slow. Updating the weights of a massive model every few steps requires significant FLOPs and time, making it difficult to scale to the largest "frontier" models. The authors ask a fundamental question: Can we achieve the same adaptation benefits more efficiently by evolving an external memory instead of changing the model itself?

Methods

The authors introduce TTME, which keeps the LLM's weights frozen and instead maintains two types of external memory:

  1. Guideline Memory: This acts as a "playbook" of operational lessons. It distills execution errors and evaluation feedback into concrete instructions (e.g., "Avoid using this specific library function because it causes a timeout").
  2. Strategy Memory: This captures high-level algorithmic "aha moments." When a modification improves a score, TTME extracts the underlying idea and stores it as a reusable strategy.

During the discovery process, a "selector" model looks at the current problem and the available strategies in memory to provide the "generator" model with specific modification instructions. This process is contrasted with weight adaptation in the figure below:

Figure 2: Weight adaptation updates model parameters through test-time RL (top). Memory adaptation updates external memory that conditions generation while keeping model weights fixed (bottom). Both retain the same underlying discovery framework.

Datasets

The researchers evaluated TTME across five challenging mathematical construction tasks:

  • Erdős’s minimum overlap problem: Searching for density functions to certify bounds.
  • Autocorrelation Inequalities (AC1, AC2, AC3): Constructing functions that satisfy specific mathematical properties.
  • Maximal Determinant Problem (Hadamard): Finding matrices with the largest possible absolute determinant.

They tested these methods using several open-weight models, including Qwen3-8B, gpt-oss-120b, and a distilled version of Qwen3-8B.

Key Insights and Ideas

  • Memory Trumps Weights: Surprisingly, distilling experience into external text-based memory was more effective than internalizing it through weight updates. The authors suggest that current LLMs already have the reasoning capabilities to use external context effectively, making weight updates redundant for these tasks.
  • The Efficiency Gap: Because TTME avoids the heavy lifting of backpropagation and optimizer states required for TTT, it is vastly more efficient.
  • Better Idea Synthesis: TTME was found to be significantly better at combining different successful ideas from previous attempts. By explicitly storing strategies, the model can "mix and match" successful approaches more reliably than a model whose weights have been subtly shifted by RL.
  • Diminishing Returns of Hybridization: When the authors tried to combine both TTT and TTME, performance actually dropped in some cases. This suggests that memory-based adaptation provides a cleaner, more focused signal for the model.

Key Results

The main takeaway is that TTME is both better and cheaper than the current state-of-the-art.

  • Superior Performance: TTME achieved the best-found scores on all five mathematical tasks, surpassing both the baseline (no adaptation) and TTT.
  • Massive Speedups: TTME reduced computational costs (FLOPs) by 82.8%–84.6% and total runtime by 47.7%–58.4% compared to TTT.
  • Rapid Convergence: As shown in the progress plots, TTME often reached the final performance level of TTT in less than half the number of steps.
Figure 1: Best-found scores versus computational cost under the TTT-Discover framework [1] with Qwen3-8B. The three settings use the same discovery framework, with TTT, with TTME replacing TTT, or with neither method. TTME achieves the best scores at low computational cost across all three tasks.

Limitations and Future Work

While the results are impressive, the authors note a few areas for further exploration:

  • Domain Specificity: The evaluation focused on mathematical code generation. It remains to be seen if TTME is as effective in other scientific domains like chemistry or biology where feedback might be less structured.
  • Memory Management: As the discovery process scales to thousands of steps, managing a growing memory without it becoming "cluttered" or "stale" will require more sophisticated distillation techniques.
  • Cross-Task Transfer: A promising future direction is "cross-task" memory—using the Guideline and Strategy memories built during one problem to jumpstart the discovery process for a completely different scientific challenge.

Summary updated

Abstract

Recent \emph{test-time training} (TTT) frameworks have enabled open-weight large language models (LLMs) to outperform inference-only frameworks powered by frontier models on scientific discovery tasks. However, TTT introduces additional computational cost for online weight updates during the discovery process, making computational efficiency an important consideration when scaling to larger models. This motivates us to explore an efficient \emph{test-time adaptation} method for scientific discovery. We introduce \emph{Test-Time Memory Evolution} (TTME), a plug-and-play test-time adaptation method, as an alternative to TTT. TTME continually updates the external memory by distilling the experience to guide subsequent discovery without updating model weights. To have a clean comparison, we evaluate TTME and TTT under the same discovery framework, replacing only the test-time adaptation method. Our evaluation shows that TTME outperforms TTT on all five mathematical tasks, while reducing computational cost by $82.8$-$84.6\%$ in FLOPs and $47.7$-$58.4\%$ in runtime. These results establish TTME as an effective and computationally efficient alternative to TTT in the evaluated discovery settings.
BibTeX
@misc{liu2026can,
  author = {Yepeng Liu and Shengmai Chen and Xuandong Zhao and Chengzhi Liu and Dawn Song and Xin Eric Wang and Yuheng Bu},
  title = {{Can Test-Time Memory Evolution Outperform Test-Time Training for Scientific Discovery?}},
  year = {2026},
  eprint = {2610.32046},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.32046},
}

Loading related content...

Figure