Pruning Agent Training Data by What Commands Actually Do

Language-model agents that work in a terminal are now trained on tens of thousands of recorded attempts, each run inside its own software container. A software container is a sealed, resettable copy of an operating system, so every run starts from the same state. These corpora are huge and highly redundant. TerminalTraj builds 32,325 Docker images and keeps 50,733 verified trajectories, where a trajectory is the full step-by-step record of one agent attempt. Much of that data teaches the same few actions many times: install a package, edit a config file, run the tests. We propose to shrink these corpora the way speech engineers once shrank voice databases for phones. The rule is to delete a recorded piece only when something else in the corpus can do its job, and to judge "doing its job" by replaying commands in the container and comparing the system state they leave behind. Surface similarity is not the test.

The approach borrows its central lesson from concatenative speech synthesis, which builds speech by stitching together recorded snippets called units. Researchers there found that deleting units that look alike, or that are rarely used, breaks the synthesizer. What works is asking whether a unit is replaceable within the selection process itself. Pruning redundant synthesis units based on static and delta unit appearance frequency cut a Mandarin database by 79.1% with only a 0.18 drop in mean opinion score, the standard listener rating of naturalness. Containers give agent data something speech never had: an exact, cheap test of functional equivalence. Two commands are interchangeable if, from the same starting snapshot, they produce the same change in the machine's state.

Core Claim

At a fixed training-token budget, choosing trajectories by execution-verified irreplaceability gives better terminal agents than three alternatives: random subsampling, dropping near-duplicates by text embedding, or reweighting by skill-label frequency. It also preserves rare domains, such as QEMU emulation and security tasks, that frequency-based methods thin out. The same score offers a principled answer to an open contradiction about failed attempts. On Data Engineering for Scaling LLM Terminal Capabilities found that keeping incomplete, unfiltered synthetic trajectories beat aggressive filtering, because failure-and-debug traces supervise recovery. SkillSynth, by contrast, succeeds with a compact, oracle-verified set. Our score keeps a failure-then-recovery segment when its state transition is rare. It drops a segment when that transition is common, or when it is a loop that makes no net state change. We predict that this resolves the tension. Neither "keep all failures" nor "keep only successes" should be best at small budgets.

Research Program

We cut each trajectory into units. A unit is one command, or a short command block, together with two things. The first is a canonicalized record of the state before it ran: which relevant files, tools, and services exist. The second is a state diff: which files changed, the exit code, a hash of normalized output, and which packages or processes appeared. Canonicalization strips timestamps, cache files, and random identifiers. It follows the declarative pre/post snapshot comparison that Agent-Diff uses for grading. The differences are purpose and object. Agent-Diff scores a task outcome; we compare units to one another. Diffs are captured cheaply by snapshotting the container's writable overlay layer at each step boundary during replay.

Equivalence is established in two stages. First, a cheap signature, combining the command family, precondition features, and diff type, groups candidate units. Second, for a sample of pairs in each group, we restore one unit's pre-state snapshot, run the other unit's command there (with paths rebound by simple substitution), and check whether the canonical diffs match. Groups that fail this check are split. The second stage is the proposal's key ingredient. It is the container analogue of the speech finding that acoustically identical units can be functionally distinct, and the reason bottom-up clustering alone failed there.

Selection then follows a diminishing-returns coverage rule. Let $K$ be the set of units already kept, and for a unit $u$ let $n_K(u)$ be the number of kept units verified equivalent to $u$. The value of adding trajectory $T$ is

$$V(T \mid K) = \sum_{u \in T} \frac{1}{1 + n_K(u)},$$

and we greedily add the trajectory with the highest $V(T \mid K)$ per training token until the budget is spent. Because each additional copy of a transition is worth less, this rule protects the long tail, much as the "award factor" in the speech method discounted units that were already chosen. Units with a zero net diff that repeat within a trajectory, which are loops, contribute nothing. Siri's on-device work motivates a second output. It found that 41% of recorded units were never selected by the synthesizer. We therefore also report how many container images are still needed once every kept unit is accounted for. This measures whether semantic pruning shrinks the environment store and not just the text.

Baselines

The naive reference point is training on the full corpus. We use TerminalTraj's trajectories with its released images. For 32B models on Terminal-Bench 2.0, the file reports TerminalTraj-32B at 22.00% TerminalTraj, Nemotron-Terminal-32B at 27.4% On Data Engineering for Scaling LLM Terminal Capabilities, and SkillSynth's Qwen3-32B at 29.6% accuracy SkillSynth. These are different corpora, so they calibrate the task rather than serve as matched comparisons. Random subsampling at each budget is the second reference.

The strongest data-side competitor is post-hoc inverse-frequency reweighting over skill labels. It is SkillSynth's generation-time idea applied after the fact: each trajectory is labeled with its skill set, and selection proceeds with inverse-frequency weights. It is strong because SkillSynth showed that guided composition beats random multi-skill composition by 3.8 points and single-skill data by 8.3 points on Terminal-Bench 2.0. It differs from our method in using labels rather than verified state effects. The second strong competitor is text-embedding near-duplicate removal over trajectories. This is the default engineering choice, and the file reports no result for it on terminal corpora. The third is success-only filtering. The fourth is keep-all-failures, which follows the Nemotron finding.

Every method selects from the same pool at the same token budgets: 10%, 25%, and 50%. Every method fine-tunes the same base model with an identical recipe and step count, and receives the same small selection-hyperparameter search, scored on a held-out split of TerminalTraj tasks. Two ablations isolate the key ingredient. The first keeps our pipeline but removes the replay check, so equivalence comes from signatures only. The second swaps in embedding similarity as the equivalence test. Beating the strongest competitor means higher Terminal-Bench 2.0 pass@1 averaged over three training seeds and repeated attempts, with the gap exceeding seed spread. It must also hold at two budgets and not reverse on Terminal-Bench 1.0.

Why not just…?

Why not just deduplicate images by Docker layer hashes? Layer sharing removes identical bytes, not identical behavior. It leaves the training data untouched and does not reduce training or evaluation compute, which Terminal-Bench 2.0 reports can reach two hours and nearly 100 million tokens for one hard-task attempt. We treat it as complementary: it is applied under every condition, and we report image counts on top of it.

Why not just drop near-duplicate trajectories by text embedding? This is the real threat. It is cheap, and it may capture most of the redundancy if commands that look alike usually act alike. The embedding-equivalence ablation settles the question directly. If it matches the replay-verified version within seed noise, the execution check is not worth its cost, and we would report that honestly. We expect it to fail mainly on rare domains, where similar-looking commands have different effects. That is why we report per-domain scores.

Why not just reweight by skill frequency as SkillSynth does? Skill labels come from an LLM and are coarse. Two "edit config" steps can have different effects, and two differently labeled steps can have the same effect. The head-to-head at matched budget measures whether state-level identity adds anything beyond labels.

Why not just keep everything, including failures? The Nemotron result makes this a legitimate reference, and it may win at 50% budget. Our claim is limited to small budgets, where we predict that keeping rare recoveries and dropping loops beats both extremes.

Proposed Experiments

The main runs use Qwen3-8B and Qwen3-14B for the full budget sweep. Their unfine-tuned baselines score 2.5% and 4.0% on Terminal-Bench 2.0, and Nemotron-Terminal reaches 13.0% and 20.2% at these sizes. The single best configuration and its strongest competitor are then confirmed at 32B. Evaluation uses Terminal-Bench 2.0, which has 89 tasks, so one task is worth about 1.1 points. For that reason we report pass@1 over repeated attempts plus pass@k, alongside Terminal-Bench 1.0. To test transfer, we repeat the selection on the SkillSynth corpus of 10,680 trajectories and ask whether an already diversity-controlled set still contains verified-redundant units.

Three checks decide the proposal. First, at 25% of tokens, our selection should match full-data TerminalTraj on Terminal-Bench 2.0 and beat embedding dedup and skill reweighting. Second, the replay-check ablation should lose more on the QEMU and Security domains than on common domains. Third, among failure segments, the recovery segments our score keeps should, when removed, cost more than an equal number of dropped failure segments restored in their place.