Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training

Anas Barakat, Souradip Chakraborty, Khushbu Pahwa, Dinesh Manocha

Abstract

Pass@k is a standard metric for verifiable tasks like coding and mathematics, measuring whether at least one of $k$ samples is correct. While directly optimizing pass@k during post-training improves multi-sample performance, it frequently degrades pass@1 (single-shot accuracy). This paper characterizes this trade-off as a result of gradient conflict induced by prompt interference. We show that pass@k objectives implicitly reweight training data toward "hard" prompts. When these prompts interfere negatively with the broader dataset, the resulting update direction rotates away from the pass@1 objective, leading to a performance "see-saw."

The Pass@k vs. Pass@1 Trade-off

In the development of reasoning models, researchers often fine-tune LLMs to maximize the probability that at least one generated solution passes a verifier. However, empirical evidence shows a recurring problem: as the model gets better at finding a correct answer in $k$ tries, its first-guess accuracy (pass@1) often suffers. This is problematic for real-world applications where latency and compute costs make single-shot reliability a priority.

The pass@k optimization trade-off: (a) shows the empirical degradation of pass@1 while pass@k improves; (b) and (c) illustrate how reweighting hard prompts (like prompt 3) rotates the update gradient away from the average pass@1 direction.

Theoretical Framework: Hard-Example Mining

The core of the issue lies in how the pass@k objective treats different prompts. Mathematically, the policy gradient for pass@k implicitly applies a weight to each prompt proportional to $(1 - p)^{k-1}$, where $p$ is the current success probability.

As $k$ increases, this weighting scheme becomes an aggressive "hard-example miner." For large $k$, the model virtually ignores "easy" prompts (where $p$ is high) and focuses almost exclusively on "hard" prompts (where $p$ is low). In some experiments, the authors found weight disparities as high as $10^{28}$, effectively turning a diverse dataset into a highly biased subset of outliers.

Prompt Interference and Gradient Conflict

Focusing on hard examples is not inherently damaging. The degradation occurs when these hard prompts exhibit negative interference with the rest of the dataset. The researchers introduced a "similarity kernel" to measure how an update for one prompt affects the performance of others.

A similarity kernel heatmap showing prompt interference. Darker regions indicate negative interference, where optimizing for one set of prompts actively harms performance on another.

If the features required to solve a "hard" prompt contradict the features needed for the majority of "easy" prompts, the pass@k update will conflict with the pass@1 update. When $k$ is large enough, the influence of these negatively interfering hard prompts dominates, rotating the total gradient until it forms an obtuse angle with the pass@1 direction.

Empirical Results on Reasoning Tasks

The theory was validated using competition-level math problems (MATH dataset) and state-of-the-art models like DeepSeek-R1-Distill-Llama-8B and Qwen-7B.

Key findings include:

  • Inner Product Flips: For Llama-8B, the pass@k reweighting caused the gradient alignment to flip from positive to negative, resulting in a strongly negative inner product (-0.613) between the pass@k update and the pass@1 objective.
  • Outlier Dominance: In the Qwen-7B model, even though "easy" prompts outnumbered "hard" ones by 21 to 1, the extreme exponential weighting caused the hard, interfering prompts to completely dictate the model's learning trajectory.

Conclusion and Future Directions

This work provides a rigorous explanation for why "training for second chances" can break an LLM's first guess. The identified mechanism—gradient conflict driven by reweighted prompt interference—suggests that future pass@k optimization methods must account for gradient alignment. Potential solutions include "gradient surgery" to mask conflicting updates or exploring alternative objective functions that balance exploration with single-shot reliability.

Summary updated

Abstract

Pass@k is a widely used performance metric for verifiable large language model tasks, including mathematical reasoning, code generation, and short-answer reasoning. It defines success if any of $k$ independently sampled solutions passes a verifier. This multi-sample inference metric has motivated inference-aware fine-tuning methods that directly optimize pass@$k$. However, prior work reports a recurring trade-off: pass@k improves while pass@1 degrades under such methods. This trade-off is practically important because pass@1 often remains a hard operational constraint due to latency and cost budgets, imperfect verifier coverage, and the need for a reliable single-shot fallback. We study the origin of this trade-off and provide a theoretical characterization of when pass@k policy optimization can reduce pass@1 through gradient conflict induced by prompt interference. We show that pass@$k$ policy gradients can conflict with pass@1 gradients because pass@$k$ optimization implicitly reweights prompts toward low-success prompts; when these prompts are what we term negatively interfering, their upweighting can rotate the pass@k update direction away from the pass@1 direction. We illustrate our theoretical findings with large language model experiments on verifiable mathematical reasoning tasks.
BibTeX
@misc{barakat2026why,
  author = {Anas Barakat and Souradip Chakraborty and Khushbu Pahwa and Dinesh Manocha},
  title = {{Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training}},
  year = {2026},
  eprint = {2602.21189},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.21189},
}
Figure