Promotion Gates for Multilingual Code Training

Training a code model with reinforcement learning on unit-test rewards regularly makes it better in Python and worse, sometimes catastrophically, in less common languages. We propose treating every checkpoint update in multilingual code training as a statistical claim that has to be proven before it is accepted. A new checkpoint replaces the current one only if it is measurably better on average and provably not worse by more than a small margin in any single language. When the evidence is inconclusive, the gate spends more evaluation where the uncertainty lies instead of guessing. The expected result is a training run whose deployed model never silently loses a language, at a small and measurable cost in average accuracy.

The need is documented. Standard group-relative policy optimization on Python-heavy code data drove LLaMA-3.1-8B to 0.0% on Perl, while a language-aware variant kept it at 14.4% (GXPO). Rebalancing pretraining data toward rare languages cost 39.70% on high-resource languages at 1B parameters, shrinking to 2.47% at 4B (Measuring the Impact of Programming Language Distribution). Current practice picks checkpoints by the average pass rate on a validation set, which hides per-language damage. Per-language evaluation sets for rare languages are also small enough that an apparent drop may just be noise.

Core Claim

Statistical gating means refusing to switch models unless a hypothesis test says the switch is an improvement. It has been shown to nearly eliminate harmful updates in classical learners. Making Learners (More) Monotone wraps a learner with McNemar's paired test, a test on the problems where exactly one of two models is correct. On MNIST with 500 random Fourier features, the wrapper cut non-monotone decisions from 27% to under 1%. That work gates on a single aggregate error and names its own failure mode: with a fixed validation set and shrinking true gains, the test loses power and the learner stays stuck on an early model.

Our claim is that multilingual code training is exactly the setting where this failure mode matters, and that three changes make gating work there:

  1. The gate tests each language separately as a non-inferiority question: is the new checkpoint no worse than the old by more than a margin?
  2. It pairs outcomes on the same problem, which is possible because benchmarks like Multilingual LiveCodeBench pose each problem in every language.
  3. It evaluates sequentially, adding validation problems only for languages whose test has not yet decided.

Concretely, let $h$ be the current champion checkpoint and $c$ a candidate. For each of $L$ languages $l$, let $\hat\Delta_l$ be the mean paired difference in per-problem pass rate ($c$ minus $h$) with standard error $\hat\sigma_l$. Let $\hat\Delta$ and $\hat\sigma$ be the same quantities pooled across languages. The candidate is promoted only if

$$\hat\Delta > z_{\alpha}\,\hat\sigma \quad\text{and}\quad \hat\Delta_l + \delta > z_{\alpha/L}\,\hat\sigma_l \ \text{ for every language } l,$$

where $\delta$ is the tolerated per-language regression, $\alpha$ the error level, and $z$ the normal quantile. The $\alpha/L$ is a Bonferroni split, so the chance that a promotion hides a regression larger than $\delta$ in some language is at most about $\alpha$.

A plain gate of this form would reject almost everything in rare languages, because their $\hat\sigma_l$ is large. The sequential step handles this: undecided languages receive more validation problems in rounds, under a fixed maximum and an alpha-spending schedule that keeps the test valid. Validation blocks are refreshed across promotions, following the fresh-holdout rule in Monotone Learning. This keeps the gate from overfitting a reused validation set over hundreds of decisions.

Research Program

We would build the gate as a wrapper around an existing multilingual reinforcement learning loop, in two modes.

Selection-only mode. Training proceeds untouched, and the gate only decides which checkpoint is deployed. This is the direct analogue of the classical wrapper.

Rollback mode. When a candidate is rejected because some language regressed, training resumes from the champion. This mirrors how update gating is used in EV3 for distillation and in Width-based Lookaheads for policy learning. Rollback can change the training trajectory itself, so it can in principle prevent collapse rather than merely avoid deploying it.

Outcomes are execution-based throughout. We run $k$ samples per problem and compute the per-problem pass fraction, since token confidence is a poor proxy for correctness in code; BaxBench finds that half of functionally correct programs are exploitable.

On the analysis side, we would characterize the cost of the gate:

  • how much average pass@1 is given up relative to ungated training;
  • how evaluation compute scales with $L$ and $\delta$;
  • whether the stuck-learner problem reported by Viering et al. reappears once sequential sampling is added.

The training setup follows GXPO: Qwen3-4B and LLaMA-3.1-8B, training problems from CodeContestPlus, and the 8-language Multi-LCB (Python, Java, C++, JavaScript, Go, PHP, Ruby, Perl) as the held-out test. The gate's validation pool is drawn from held-out CodeContestPlus problems, never from the test set. To check that the gated model has not quietly lost languages outside the training eight, we also evaluate the final models on McEval (40 languages) and BabelCode's BC-HE and BC-MBPP.

Baselines

Naive reference. Deploy the final checkpoint of standard group-relative policy optimization, as practitioners commonly do. This is the regime in which GXPO reports the Perl collapse to 0.0% on LLaMA-3.1-8B.

Greedy argmax. Select the checkpoint with the highest average validation pass@1, using the same validation problems and the same total evaluation budget as the gate. This is the direct target of the "raw argmax" critique in Using Language and Translation Models to Select the Best among Outputs from Multiple MT Systems. That paper found that requiring a significant score advantage (a Kruskal–Wallis test) gave 2–6% more top-ranked outputs than picking the highest single score.

Strongest existing method: GXPO. GXPO builds language awareness into the training objective. It reports +4.1% average pass@1 over standard group-relative optimization for Qwen3-4B and +3.2% for LLaMA-3.1-8B on Multilingual LiveCodeBench v6. We would rerun it with its published settings on the same data and seeds, and apply the gate on top of both standard optimization and GXPO.

Beating GXPO means one of two things:

  • the gated run shows fewer languages with a test-set regression greater than $\delta$ relative to the starting model, at an average pass@1 within one point of ungated GXPO; or
  • gating standard optimization matches GXPO's per-language floor without GXPO's hand-tuned advantage weights, which that paper leaves as an open tuning problem.

Per-language test set sizes for Multilingual LiveCodeBench are not given in the file. The file has no reported per-language confidence intervals, so the $\delta$ we can resolve will be fixed after measuring them.

Ablations. Each isolates one ingredient:

  • Pooled McNemar gate in the style of Viering et al., on the average only. This removes per-language stratification.
  • Per-language gate with a fixed validation set. This removes sequential sampling and directly tests whether the stuck-learner failure appears.
  • Selection-only versus rollback. This tests whether gating changes training dynamics or only deployment.

All variants share the same training compute. Evaluation compute is reported separately and matched between the argmax baseline and the gate.

Why not just…?

Why not just use GXPO, which already prevents collapse? GXPO does lift Perl from 0.0% to 14.4%, and it is a real threat to the need for this proposal. It changes the objective, however, and provides no check on which checkpoint is safe to ship. Its advantage weights are set empirically, and the file notes that poor weighting can cause high-resource regression or low-resource starvation. The experiment settles this directly. If GXPO with argmax selection already shows no language regressing beyond $\delta$ on Multilingual LiveCodeBench and McEval across seeds, the gate adds nothing to GXPO, and we would report that result. The gate would then be justified only as a cheaper substitute for GXPO on top of standard optimization.

Why not just pick the checkpoint with the best average validation score? An average can rise while one language falls. Per-language argmax is not an option, because a single model must be deployed. The larger problem is noise. With small rare-language validation sets, argmax both misses real regressions and chases lucky ones, the pattern behind the 27% non-monotone decisions reported by Making Learners (More) Monotone. The equal-budget argmax baseline measures exactly this.

Why not just evaluate on a much larger validation set? Given unlimited evaluation, a larger set approaches the gate's behavior, which makes it the honest competitor on compute. Rare-language problem pools with working test harnesses are limited, however; the file lists sandbox construction for low-resource languages as a known bottleneck. Uniform enlargement also spends evaluation on Python, where decisions are already clear. We compare the gate against uniform evaluation at matched budget. If uniform sampling achieves the same regression control at the same cost, the sequential component is unnecessary, and the contribution reduces to per-language non-inferiority testing.

Proposed Experiments

The decisive measurement is the number of languages, averaged over seeds, whose final Multilingual LiveCodeBench pass@1 falls more than $\delta$ below the starting model. We report it together with average pass@1 and total evaluation compute, for each baseline and gate variant on both base models. A second check counts the fraction of accepted promotions that turn out to be regressions when judged on a large, separately held-out pool. This is the multilingual analogue of the 27%-versus-under-1% measurement.

Three further checks complete the evaluation:

  • Transfer beyond the training languages. McEval's 40 languages test whether guarantees on the gated eight carry over to languages the gate never saw, or whether regressions move there.
  • Rollback against the collapse regime. We compare rollback and selection-only runs on LLaMA-3.1-8B under standard optimization. If rollback keeps Perl well above 0.0% without GXPO's objective changes, gating is shown to be a training intervention and not just a deployment filter.

The proposal fails if gated runs show no fewer per-language regressions than GXPO with argmax selection. It also fails if their average pass@1 falls more than a point below ungated training at every $\delta$ where the regression guarantee holds.