When a language model reasons step by step in Thai, Swahili or Arabic, each reasoning step costs several times more tokens than the same step in English. That happens because tokenizers built mostly for English break those languages into many small fragments. We also know that longer reasoning is not always better. Accuracy rises with chain-of-thought length, peaks, and then falls as errors pile up, an "inverted U." This proposal asks a simple, untested question: do errors pile up per reasoning step or per token? If per token, a high-fragmentation language should hit its accuracy peak after fewer steps than English. The well-documented advantage of language-specific models, which often come with tokenizers built for the target language, would then partly be the effect of moving that peak back within reach. If per step, native tokenizers only buy cheaper compute, and the reasoning gap has some other cause.
Answering this needs an evaluation that existing multilingual reasoning benchmarks cannot provide. They are translations of English sets such as GSM8K. Benchmark leakage is known to cross language barriers, and current leakage detectors fail on reasoning models. So any measured "optimal length" on those sets could reflect recall of a memorized answer rather than reasoning. We therefore build a multilingual, procedurally generated reasoning suite with known ground-truth step counts. On it we measure accuracy as a function of reasoning budget across languages and across matched pairs of models that differ mainly in tokenizer.
Core Claim
The central hypothesis is that the per-language peak of the accuracy-versus-reasoning-length curve, measured in ground-truth reasoning steps, moves earlier as tokenizer fertility rises. Fertility here means the number of tokens produced per word. We further hypothesize that vocabulary expansion of the kind used by SeaLLMs moves the peak later by roughly the factor by which it cuts fertility. To make this falsifiable, write accuracy at step budget $s$ in language $\ell$ as approximately $(1-\epsilon)^{c_\ell s}\,g(s)$. In this expression, $g(s)$ is the benefit of decomposing the problem into $s$ steps, $\epsilon$ is an error rate per unit of generated length, and $c_\ell$ is the length of one step in units where errors accrue.
Under a per-step error model, $c_\ell = 1$ for every language, so the peak step $s^*$ is independent of language. Under a per-token model, $c_\ell$ is proportional to the fertility $f_\ell$, and $s^*$ shrinks as $f_\ell$ grows. This functional form is the error-accumulation-versus-decomposition trade-off that When More is Less: Understanding Chain-of-Thought Length in LLMs uses to derive the inverted U on English tasks. We only add the question of which unit the error accrues in. Intermediate results, where $c_\ell$ grows with fertility but sub-linearly, are informative too, and the experiment estimates $c_\ell$ directly.
Where This Sits
Three papers motivate the question. When More is Less establishes the inverted U and shows that the optimal length depends on task difficulty and model size, but it works only in English. The Token Tax shows that across 16 African languages on AfriMMLU, fertility explains 20–50% of accuracy variance. It also finds that reasoning models narrow the English–African gap from about 25 points to 12–14. It never asks whether longer deliberation in high-fertility languages hits the overthinking cliff sooner, and that is exactly our question. Crosslingual Reasoning through Test-Time Scaling supplies the main method. Its "budget forcing" truncates chains or appends "Wait" to lengthen them, and its "language forcing" fixes the language of the reasoning trace. It reports a +9.4% MGSM gain for s1-14B going from 0.5k to 8k thinking tokens, and roughly 3.5× more compute to reason in Swahili than in French. It does not locate per-language peaks, and it does not control for contamination.
Two papers constrain the design. Data Contamination Can Cross Language Barriers shows that fine-tuning on translated benchmarks inflates scores without tripping overlap or perplexity detectors; LLaMA3-8B on MathQA went from 42.01% to over 95%. On The Fragility of Benchmark Contamination Detection in Reasoning Models shows that detectors fall to 50–58% AUROC on reasoning models contaminated with chain-of-thought data. Together they rule out the cheap design of running budget sweeps on MGSM or translated GSM8K and filtering out leaked items. Memorized items would look like "solved at short length," which would bias the measured peak early, and we could not reliably remove them.
Two papers supply the evaluation substrate. DyVal generates reasoning problems from directed acyclic graphs with controllable depth and finds that depth, not width, is the main failure axis. BeyondBench generates solver-verified algorithmic tasks and documents sharp accuracy drops as depth grows. Both are English-only and neither measures reasoning length. Cards Against Contamination: TCG-Bench is the closest multilingual contamination-resistant benchmark: it finds a 47.4% English–Arabic gap for 32B models, but it tracks only game outcomes, never reasoning length or tokenization.
On the language-specific side, SeaLLMs gives the natural experiment we need. It took LLaMA-2, expanded the vocabulary so that Thai fertility fell from 4.29 to 1.57, and reported Thai translated-GSM8K accuracy of 62.0 versus 35.8 for ChatGPT-3.5. However, it was evaluated on a translated, potentially leaked benchmark. ArabicMMLU shows that Arabic-native Jais-chat 30B reaches 62.3% and beats larger general models. Language-Mixed Chain-of-Thought shows that letting Korean reasoning switch to English for formal steps helps a great deal. Our design tests one mechanistic reading of these results, the token budget explanation, and does not assume it.
Research Program
The first deliverable is a multilingual version of the DyVal generator, built from the implementation in PromptBench. We cover arithmetic graphs, boolean and deductive logic, and reachability, extended past DyVal's D4 depth so that the downward side of the curve is reachable. Only the natural-language templates are translated, by native speakers. Every evaluated instance is freshly sampled, so no instance can have appeared in training data.
The languages are English, French, Vietnamese, Thai, Arabic, Korean, Swahili and Telugu. This spans low to very high fertility and covers languages with native or adapted models. Each instance has a known number of intermediate node evaluations, which serves as a language-neutral step count. We parse the intermediate values that appear in a reasoning trace to count how many steps the model actually executed.
The second deliverable is a set of budget-sweep curves. For each model, language and depth, we vary the reasoning budget with budget forcing over a grid running from immediate answering to very long chains. We record accuracy against three quantities: generated tokens, generated characters, and executed ground-truth steps. We then fit the model above to estimate $c_\ell$ per language and test whether it tracks measured fertility.
The third deliverable uses matched model pairs that hold the base model roughly fixed while changing the tokenizer or pre-training language. LLaMA-2 is paired with SeaLLM for Thai and Vietnamese. A general multilingual model of comparable size is paired with Jais for Arabic. The s1 / Qwen2.5 family, run under language forcing, separates the cost of reading the prompt in the native language from the cost of writing the trace in it. We also include KO-REAson as a language-mixed reference for Korean.
Baselines
The naive reference point is native-language chain-of-thought at the model's default length, with no budget control. This is how SeaLLMs and ArabicMMLU-style evaluations are usually run. Every curve reports this point, and the cross-language gap at default length is the quantity that later explanations must account for. No reported result was found on Lacuna for any of these models on a multilingual DyVal-style suite, so the first published numbers will come from this study.
The strongest existing competitor for "reasoning in a non-English language" is English chain-of-thought: reason in English, answer in the target language. Language models are multilingual chain-of-thought reasoners found that this consistently beat native reasoning; it reported 55% average MGSM for PaLM-540B. Yong et al. found that s1 models do this spontaneously ("quote-and-think"). We run it through language forcing with the same models, instances and budget grid as native reasoning. Any claim that native tokenizers help reasoning must hold up against this baseline at matched compute, measured in total tokens and in wall-clock FLOPs, not just at matched step count.
The strongest existing method for choosing reasoning length is length-aware selection over parallel samples. This covers Length-Filtered Vote from When More is Less, short-m@k from Don't Overthink it, and Shortest Majority Vote from Revisiting the Test-Time Scaling of o1-like Models. No reported numeric result was found on Lacuna for these methods on multilingual tasks; the file states only that Length-Filtered Vote beats self-consistency on GPQA, without a figure. For the budget-scaling side, the reference number is Yong et al.'s +9.4% on MGSM for s1-14B from 0.5k to 8k tokens. We run all selection methods with the same sample count $k$ per instance. A per-language budget rule derived from our fitted $c_\ell$ beats them if it matches their accuracy at lower token cost, or exceeds it at equal cost, in the high-fertility languages.
The key ablation isolates fertility from everything else that differs between languages. Within the LLaMA-2/SeaLLM pair, Thai fertility differs by about 2.7× while the base weights are shared. If the step-measured peak for Thai moves later by a comparable factor, while Vietnamese moves less, that isolates the tokenizer. As a weaker within-English control, we artificially raise English fertility by inserting separators that force finer token splits. This shifts the input distribution, so we treat it only as corroboration.
Why not just…?
Why not just extend Yong et al.'s budget sweeps on MGSM? MGSM is 250 translated GSM8K items, and leakage of GSM8K across languages is exactly what Data Contamination Can Cross Language Barriers demonstrates. Leaked items are answered correctly at short budgets, which pulls the apparent peak earlier and could mimic the per-token effect we are testing. Fragility shows we cannot filter those items out reliably. MGSM also lacks controlled depth, so the step axis is unavailable. We will still run MGSM alongside the generated suite: a difference between the two sets of curves is itself evidence of how much contamination distorts length studies.
Why not just divide the token budget by fertility and be done? That is the per-token hypothesis stated as a rule, and it may well be right. If the fitted $c_\ell$ tracks fertility closely, the finding is that native tokenizers buy reasoning headroom only through cheaper steps. That would be a useful and deflationary answer to the monolingual-advantage claim. The experiment does not assume this rescaling; it tests it against the per-step alternative and against intermediate cases, for example if comprehension errors in the native prompt dominate.
Why not just reason in English and ignore native tokenization? This is a real threat. If English reasoning at matched total compute closes the gap on generated tasks, then trace-side fertility does not matter and the native-model advantage reported by SeaLLMs is about understanding the prompt, not about reasoning. The language-forcing arm settles this directly, because it holds the prompt fixed and varies the trace language. A residual gap under English reasoning that grows with depth would point to comprehension; a gap that disappears would point to the trace.
Decisive Checks
The first decisive check is whether, on the generated suite, step-measured peak location falls monotonically with fertility across the eight languages, beyond what model differences explain. The second is whether SeaLLM's Thai peak sits later than LLaMA-2's Thai peak by a factor near the 4.29→1.57 fertility reduction. The third is whether the default-length cross-language gap shrinks once each language is run at its own peak budget.
If all three hold, a per-language budget rule becomes a cheap, actionable fix, and native tokenization has a concrete mechanism. If peaks coincide in step units across languages, the per-step model holds. In that case, the "token tax" on reasoning is purely economic, and the remaining gap must be sought in comprehension or in contamination of translated benchmarks, which the MGSM-versus-generated comparison will quantify.
References
- When More is Less: Understanding Chain-of-Thought Length in LLMs
- The Token Tax: Systematic Bias in Multilingual Tokenization
- Crosslingual Reasoning through Test-Time Scaling
- Data Contamination Can Cross Language Barriers
- On The Fragility of Benchmark Contamination Detection in Reasoning Models
- DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks
- BeyondBench: Contamination-Resistant Evaluation of Reasoning in Language Models
- Cards Against Contamination: TCG-Bench for Difficulty-Scalable Multilingual LLM Reasoning
- SeaLLMs -- Large Language Models for Southeast Asia
- ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic
- Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought
- Language models are multilingual chain-of-thought reasoners
- Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoning
- Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?