What to Practice Next? Latent Profile-Guided Task Selection for Skill-Evolving LLM Agents

Qijin Zhang, Hanzhen Wang, Yan Huang, Zhongxiang Dai, Shuang QiuarXiv 2026 · arXiv

TL;DR

This paper introduces SPG-Select, a framework that helps LLM agents decide which tasks to practice to improve their skills most efficiently. By using Multidimensional Item Response Theory (MIRT) to map an agent's latent strengths and weaknesses from simple pass/fail outcomes, the system prioritizes tasks that target the agent's specific "proficiency gaps," significantly outperforming random task selection across various benchmarks.

What Problem Does This Paper Address?

Large Language Model (LLM) agents are increasingly capable of "self-evolution"—improving their performance over time by reflecting on past experiences and building skill libraries. However, most current agents follow a fixed or random practice schedule.

This is inefficient for two reasons:

  1. Multidimensionality: Different tasks require different skills (e.g., a "cleaning" task in a virtual house requires different logic than a "heating" task). A single scalar progress score doesn't capture which specific skill needs work.
  2. Nonstationarity: As an agent learns, the "value" of a task changes. A task that was once a great learning opportunity becomes trivial once mastered.

The authors tackle the challenge of finite-budget task selection: given a limited number of practice rounds, how should an agent choose tasks to maximize its eventual success on unseen, held-out challenges?

Methods

The authors propose Skill Profile-Guided Task Selection (SPG-Select). The framework operates in several key stages:

  • Latent Profile Estimation (MIRT): Instead of directly measuring skills, the authors use Multidimensional Item Response Theory. By observing binary outcomes (success/failure) across different tasks, the model infers a $K$-dimensional latent "profile" ($\hat{s}_t$) for the agent. This separates task difficulty from the agent's specific proficiencies.
  • Surrogate Progress: The framework defines "progress" as the temporal difference between consecutive profile estimates. It doesn't just look for success; it looks for tasks that cause the agent's internal skill estimate to shift.
  • Gap-Weighted UCB: The core policy (SPG-Select-R) uses a "Upper Confidence Bound" approach common in bandit problems. Crucially, it weights predicted progress by the "proficiency gap"—the difference between current skill and mastery. If an agent is weak in "spatial reasoning," tasks that improve that dimension are prioritized.
  • Handling Nonstationarity: Because the agent is constantly evolving, old data becomes "stale." The authors implement a periodic restarting strategy (the "-R" in the name) to reset statistics and adapt to the agent's new, improved state.

Datasets

The researchers evaluated their framework on two primary benchmarks using three different "skill-evolution" backends (SkillRL, ExpeL, and Shared Reflexion):

  1. ALFWorld: A text-based environment for household tasks (e.g., "put a clean sponge in the cabinet"). It includes six task families: Pick, Look, Clean, Heat, Cool, and Pick2.
  2. WebShop: A simulated e-commerce environment where agents must search for, inspect, and purchase products based on complex natural language instructions.

Key Insights and Ideas

  • Inference from Binary Feedback: One of the most elegant parts of the paper is using MIRT to turn "Yes/No" success signals into a nuanced, multidimensional skill vector. This allows the agent to "know what it doesn't know" without needing manual skill labeling.
  • The "Surrogate" Advantage: The model prioritizes surrogate progress (changes in the internal model's belief) rather than just immediate success. This encourages the agent to tackle challenging tasks that are "just right" for its current level.
  • Restarting is Essential: The authors found that simply accumulating data forever (Full History) actually hurt performance compared to periodic restarts. This confirms that in self-evolving systems, the agent's past self is a poor predictor of its current learning needs.

Key Results

The authors compared SPG-Select-R against uniform (random) task sampling. The results were consistently positive:

  • ALFWorld: SPG-Select-R achieved gains of 1.49 to 13.18 percentage points in success rates over uniform sampling across different backends.
  • WebShop: The framework saw improvements of 0.93 to 3.13 percentage points.
  • Efficiency: In five out of six backend-benchmark combinations, SPG-Select-R achieved these higher success rates while using 8% to 42% fewer API calls during evaluation, suggesting it builds more robust and concise skill memories.
  • Theoretical Grounding: The paper provides a high-probability dynamic pseudo-regret bound, mathematically proving that the task allocation strategy is sound under certain linear assumptions.

Limitations and Future Work

  • Warmup Requirements: The system requires an initial "warmup" phase of random sampling to calibrate the MIRT model. In extremely low-budget scenarios, this overhead might be significant.
  • Linearity Assumptions: The theoretical proofs rely on a predictable linear surrogate model. Real-world LLM learning dynamics are likely non-linear and more chaotic, which might affect the tightness of the regret bounds.
  • Fixed Task Pool: Currently, the agent selects from a fixed pool of tasks. A natural next step would be integrating this with generative curriculum learning, where the agent not only selects tasks but also invents new ones to target its specific skill gaps.
  • Dimensionality ($K$): The choice of how many skill dimensions ($K$) to track is currently a hyperparameter. Automating the discovery of these dimensions based on task embeddings would make the system even more autonomous.

Summary updated

Abstract

Task selection shapes the experiences available for skill evolution in LLM agents, yet fixed practice schedules overlook differences in skill requirements and evolving task value. Most existing methods, however, rely on fixed schedules or scalar measures of progress, leaving nonstationary skill demands unaddressed. We propose SPG-Select, a framework for allocating limited practice budget using a multidimensional profile inferred from binary outcomes via MIRT. The framework predicts surrogate progress, defined as the difference between consecutive profile estimates, and prioritizes progress in dimensions with larger estimated proficiency gaps. Our SPG-Select-R policy combines these predictions with uncertainty-based exploration and periodic restarting to accommodate nonstationarity. We evaluate SPG-Select on ALFWorld and WebShop with three skill-evolution backends. In the reported runs, SPG-Select-R achieves higher mean held-out terminal success than uniform sampling in all six backend-benchmark combinations, with gains of 1.49-13.18 percentage points on ALFWorld and 0.93-3.13 points on WebShop. Under a predictable linear surrogate model and boundedness assumptions, we establish a high-probability dynamic pseudo-regret bound for surrogate task allocation in terms of parameter path variation.
BibTeX
@misc{zhang2026what,
  author = {Qijin Zhang and Hanzhen Wang and Yan Huang and Zhongxiang Dai and Shuang Qiu},
  title = {{What to Practice Next? Latent Profile-Guided Task Selection for Skill-Evolving LLM Agents}},
  year = {2026},
  eprint = {2611.18818},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.18818},
}
Figure