A-RSI: Adaptive Task Refresh for Zero-Data Multimodal Self-Improvement

Yu Liu, Youxing Li, Yuedong Tan, Zhiwei Yang, Zihan Su, Hanqi Feng, Shaojie Zhang, Libin Liao, Zihao Wan, Chenxu Wu, Cong Cao, Fangfang Yuan, Yanbing Liu, Ziheng Chen, Pei Fu, Radu Timofte, Zongwei Wu, Nicu Sebe, Barnabas Poczos, Jian LuanarXiv 2026 · arXiv

TL;DR

A-RSI (Adaptive Recursive Self-Improvement) is a new framework that allows multimodal models to self-improve without any human-labeled data by programmatically generating verifiable tasks. Its key innovation is an "adaptive refresh" rule that monitors training logs to detect exactly when a model has exhausted the learning signal from its current task pool, triggering a refresh only when needed to maximize efficiency.

What Problem Does This Paper Address?

Recursive self-improvement (RSI) is the "holy grail" of AI: a model that generates its own training data to get smarter. However, current RSI methods face two major hurdles:

  1. The "When" Problem: Most methods refresh training data on a fixed schedule or based on external test scores. This is inefficient—refreshing too often wastes compute on new tasks, while refreshing too slowly leaves the model spinning its wheels on "saturated" data it has already mastered.
  2. The "Truth" Problem: Many systems use "self-consistency" (majority voting) to label self-generated data. If the model is consistently wrong, it ends up training on its own hallucinations, reinforcing its errors.

A-RSI tackles these by separating the decision to refresh (based on internal training dynamics) from the source of truth (based on verifiable, program-computed answers).

Methods

The A-RSI pipeline consists of three main stages, as illustrated in the overview below:

Overview of the A-RSI workflow, showing programmatic task generation, the training loop, and the log-triggered refresh mechanism.
  • Programmatic Task Construction: Instead of asking a model to "write a math problem," the authors use model-written programs. These programs generate a geometric configuration or data table, render it into an image, and—crucially—compute a ground-truth answer from the underlying code. This ensures the labels are 100% reliable.
  • Adaptive Refresh Rule: During training (using Group Relative Policy Optimization or GRPO), A-RSI monitors the "relative-advantage signal." If most responses to a prompt receive the same reward, that prompt is no longer providing a useful gradient for learning. When the number of "informative" prompts drops below a threshold (saturation), the controller triggers a pool refresh.
  • Zero-Data Loop: The entire process uses no external training examples, human-written labels, or validation scores to make decisions.

Datasets

  • Training: A reservoir of 16,000 programmatically generated tasks covering geometry, charts, tables, and counting.
  • Evaluation: The authors use a primary suite of eleven benchmarks (referred to as Macro11), including:
  • Visual Reasoning: MathVerse, MathVision, MMMU, MMMU-Pro (multiple versions), and VisNumBench.
  • Specialized Tasks: ChartQA, PhyX (physics), MMSI (spatial intelligence), and SCEMQA-MC (science).

Key Insights and Ideas

  • Learning to Finish vs. Learning to Solve: A major "aha moment" in the paper is the distinction between a model getting smarter and a model simply becoming more concise. Many RSI gains are actually "completion gains"—the model learns to stop repeating itself and finish its Chain-of-Thought (CoT) within the token limit. A-RSI explicitly audits these to ensure real reasoning improvements.
Motivation for A-RSI, distinguishing between simple completion gains (shorter CoT) and true correctness gains.
  • Training Logs as a Compass: The authors show that you don't need a test set to know when to stop training. By watching the variance in rewards within a batch, the system can "feel" when a dataset has stopped being challenging.
  • Simplicity Wins: Surprisingly, complex "frontier" selection strategies (trying to pick tasks at the edge of the model's ability) didn't outperform simple random sampling from the programmatic library, provided the timing of the refresh was handled correctly.

Key Results

The authors tested A-RSI on the Qwen3-VL-8B-Instruct model.

  • Performance Boost: A-RSI raised the Macro11 average from 57.77 to 61.05 using only 800 fresh tasks.
  • Efficiency: Compared to a "Continuous" refresh strategy (refreshing every window), A-RSI achieved higher accuracy (+0.40 points) while using 75% less new data (800 vs 3,200 tasks).
  • Robustness: The gains were not just due to shorter outputs. In an audit where the token budget was doubled, A-RSI retained 80% of its performance gain, whereas a baseline competitor (MM-Zero) saw its gains drop significantly.
Comparison of refresh policies, showing that A-RSI achieves higher accuracy with significantly fewer fresh tasks than continuous refresh.

Limitations and Future Work

  • Scope of Generators: The current programmatic library is fixed and doesn't cover all possible multimodal domains (e.g., it lacks function plots or complex natural scenes).
  • Single-Run Data: Most results are based on single training runs per policy. While the gains are consistent, more extensive replication across different random seeds would strengthen the statistical claims.
  • Future Directions: The authors suggest exploring "low-entropy" models and expanding the programmatic library to more diverse task families to see if the "random wins" insight holds at larger scales.

Summary updated

Abstract

Recursive self-improvement requires models not only to acquire useful training experience, but also to decide when their current experience has ceased to be useful. Existing methods often focus on what experience to generate next, while paying less attention to when fresh experience should be introduced. Many also construct training targets through self-consistency or majority voting, making supervision depend on the model's own uncertain predictions. We address these two questions separately: training dynamics determine when to acquire new experience, while the answers to new tasks remain independently checkable. A-RSI monitors reward variation and accuracy in training logs to refresh a task pool when its learning signal declines, without consulting validation or test scores. A static library of model-authored programs produces multimodal questions and derives their reference answers; we specify the task-generation contract, automatically verify chosen properties of the reward and refresh rules, and separately audit the generated answers. The training loop uses no external training examples or human-written answer labels. On the primary eleven-benchmark suite, A-RSI raises the performance average from 57.77 to 61.05 with 800 fresh tasks.
BibTeX
@misc{liu2026rsi,
  author = {Yu Liu and Youxing Li and Yuedong Tan and Zhiwei Yang and Zihan Su and Hanqi Feng and Shaojie Zhang and Libin Liao and Zihao Wan and Chenxu Wu and Cong Cao and Fangfang Yuan and Yanbing Liu and Ziheng Chen and Pei Fu and Radu Timofte and Zongwei Wu and Nicu Sebe and Barnabas Poczos and Jian Luan},
  title = {{A-RSI: Adaptive Task Refresh for Zero-Data Multimodal Self-Improvement}},
  year = {2026},
  eprint = {2610.38568},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.38568},
}
Figure