Terminal-World: Scaling Terminal-Agent Environments via Agent Skills

Zihao Cheng, Hongru Wang, Zeming Liu, Xinyi Wang, Xiangrong Zhu, Yuhang Guo, Wei Lin, Jeff Z. Pan, Yunhong WangarXiv 2026 · arXiv

TL;DR

Terminal-World is a fully automated data synthesis pipeline that trains language models to become highly proficient terminal agents capable of executing complex command-line tasks. By using structured "agent skills" as a foundational building block, it co-derives realistic task instructions, executable environments, and optimized trajectories. The resulting models outperform state-of-the-art baselines on terminal benchmarks while requiring up to 85× less training data.

What Problem Does This Paper Address?

Building LLM-based agents that can interact directly with a Bash terminal (similar to tools like Claude Code) is a major frontier in AI automation. However, training these agents requires high-quality data triples: a task instruction, a matching executable environment (files, dependencies, configurations), and a multi-turn trajectory of correct commands.

Manually curating this data is prohibitively expensive, and existing automated methods suffer from three core limitations:

  1. Limited Tasks: They bootstrap from narrow human-defined seeds or specific GitHub repositories, failing to capture real-world complexity.
  2. Environment Misalignment: Environments are often retrofitted around tasks after the fact, leading to fragile or unrealistic setups.
  3. Trajectory Inefficiency: Without explicit procedural guidance during data collection, teacher models rely on unguided exploration, creating training data filled with redundant commands and suboptimal paths.

Methods

Terminal-World solves these issues by introducing agent skills as the central synthesis primitive. A skill explicitly encodes three dimensions: what to accomplish, when to apply it (preconditions/states), and how to execute it.

The pipeline operates in four distinct stages:

  1. Agent Skill Collection: The authors filter 10,000 raw skills from open-source repositories down to 1,000 high-quality skills across 12 categories. To scale complexity, they compose individual skills into "skill teams" (for multi-role workflows) and "skill graphs" (for cross-domain pipelines).
  2. Task Generation: Primitives are paired with diverse user personas to act as context multipliers. The pipeline then generates a unified task specification containing instructions, environment blueprints, evaluation criteria, and execution guidelines.
  3. Environment Building: To instantiate the environment blueprint, a multi-agent Generate-Verify-Repair (GVR) loop programmatically creates initial files, generates setup scripts, and builds robust pytest verifiers to ensure the environment behaves correctly.
  4. Trajectory Collection: A teacher model (DeepSeek-V3.2) executes the task. Crucially, it is guided by the skill's procedural execution guidelines to ensure the resulting trajectory is concise and optimal. The guidelines are stripped out before final student training so the agent learns autonomous planning.
Overview of the Terminal-World pipeline

Datasets

Using this pipeline, the authors constructed a training dataset of 5,723 high-fidelity terminal-agent environments, spanning 104 file types and 1,939 distinct Bash commands.

For evaluation, they tested their models across 6 diverse benchmarks:

  • Terminal-Bench 2.0: A native terminal-agent coding benchmark consisting of 89 tasks.
  • AIME 24 / AIME 25: High-difficulty math reasoning benchmarks converted into terminal tasks where agents must write and execute Python scripts to solve problems.
  • DABench / TableBench: Table-based data analysis benchmarks.
  • BIRD: A large-scale SQL-based data analysis benchmark deployed within a local SQLite terminal environment.

Key Insights and Ideas

  • Skills as a Unified Anchor: Instead of generating tasks in a vacuum, anchoring the generation process to a pre-aligned skill ensures that the environment, task, and solution path are inherently synchronized from the start.
  • The Crucial Role of Failure Trajectories: In ablation studies, the authors found that training on both successful and failed trajectories (where the agent makes an error but corrects it) is vital. Removing failure trajectories degraded performance more than reducing the overall dataset size, proving that error-recovery processes offer high-quality supervision signals.
  • Uniform Penalization Hurts: Attempting to suppress unsuccessful trajectories using a negative SFT loss actually decreased performance. This is because failed trajectories still contain mostly correct intermediate steps; penalizing the entire sequence indiscriminately suppresses good terminal behaviors.

Key Results

The resulting models, Terminal-World-8B/14B/32B, achieved remarkable sample efficiency and performance gains:

  • Data Efficiency: On Terminal-Bench 2.0, Terminal-World-32B achieved a 31.5% Pass@1, surpassing Nemotron-Terminal-32B by +4.5% absolute points, despite using only 1.2% of the training data (5.7K vs 490.5K samples).
  • Behavioral Efficiency: Terminal-World agents exhibited much cleaner command-line behavior. They required significantly fewer steps and total commands to solve tasks while maintaining a lower command error rate than both the baselines and their own teacher model.
  • Cost Effectiveness: Due to the automated harness and efficient guidance, the entire dataset cost only $999.59 to generate, averaging a mere **$0.17 per trajectory**.
Behavior statistics on Terminal-Bench 2.0 showing superior interaction efficiency

Limitations and Future Work

The authors identified four primary failure modes that still plague their terminal agents:

  1. Context Window Overflow: In long trial-and-error tasks, massive terminal outputs fill the context window, triggering history compression. This causes the agent to forget prior attempts and fall into repetitive loops.
  2. Execution Deadlocks: Agents occasionally get stuck in interactive terminal prompts (like a heredoc) and fail to execute escape sequences effectively.
  3. Premature Completion: Agents sometimes rely too heavily on superficial feedback (e.g., a compiler warning count hitting zero) and declare success without verifying if the underlying task requirements were semantically met.
  4. Task Substitution: When faced with highly complex logical objectives, agents occasionally take shortcuts—such as using a simple grep string match instead of writing a robust parsing script—circumventing the core task.

Summary updated

Abstract

Terminal agents extend Large Language Models with the ability to execute tasks directly in command-line environments, but their progress is bottlenecked by the scarcity of high-quality training data. Existing approaches bootstrap from partial sources such as human-defined seeds or GitHub repositories to instantiate one component and then complete the rest, producing tasks confined to narrow seed distributions, environments misaligned with task semantics, and inefficient trajectories from unguided exploration. To address these limitations, we introduce Terminal-World, a fully automated pipeline that uses agent skills as the central synthesis primitive, which jointly encode what to accomplish, when to apply (preconditions and environment state), and how to execute, enabling task instructions, environments, and teacher trajectories to be co-derived. To further broaden the synthesis space, Terminal-World composes skills into skill teams and skill graphs for multi-role and cross-domain task synthesis. Using this pipeline, we construct 5,723 training environments and train Terminal-World-8B/14B/32B, evaluated across 6 benchmarks where the Terminal-World series consistently outperforms terminal-agent baselines. Notably, using the same teacher model and only 1.2% of the training data, Terminal-World-32B surpasses Nemotron-Terminal-32B on Terminal-Bench 2.0 by +4.5 Pass@1 (31.5) and achieves 43.8 Pass@3.
BibTeX
@misc{cheng2026terminal,
  author = {Zihao Cheng and Hongru Wang and Zeming Liu and Xinyi Wang and Xiangrong Zhu and Yuhang Guo and Wei Lin and Jeff Z. Pan and Yunhong Wang},
  title = {{Terminal-World: Scaling Terminal-Agent Environments via Agent Skills}},
  year = {2026},
  eprint = {2605.20876},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.20876},
}

Loading related content...

Figure