Inherited, Not Invented: Tracing the Provenance of Wait in Reasoning Models

Zhizhuo Kou, Sizai Hou, Long Chen, Jian Yang, Junyu Luo, Jingshu Peng, Wenkai Zhang, Sirui Han, Yike GuoarXiv 2026 · arXiv

The "aha moment" has become a central trope in the narrative of modern reasoning models. We’ve all seen the traces: a model starts solving a math problem, stops, writes "Wait, is that correct?", and then corrects its own logic. Conventional wisdom suggests that Reinforcement Learning (RL) "discovers" this reflective behavior, coining these markers as it learns to maximize rewards.

A new paper from researchers at HKUST, Peking University, and Huawei Noah’s Ark Lab challenges this narrative. Their finding? The word "wait" is inherited, not invented.

TL;DR

Reinforcement Learning (RL) doesn't teach models to say "wait"; it merely amplifies and reshapes markers already present in pre-training or fine-tuning data. By tracing the "provenance" of these markers across 120 model checkpoints, the authors show that models can achieve massive reasoning gains without ever using the word, debunking the idea that RL "coins" reflective language.

What Problem Does This Paper Address?

As models like DeepSeek-R1-Zero popularized the "aha moment," researchers began using keywords like "wait" or "recheck" as proxies for measuring a model's reasoning ability. However, it was unclear whether RL was actually teaching the model the concept of doubt or if the model was simply latching onto high-reward linguistic patterns it had already seen.

The authors set out to determine the true origin of these revision markers. Does RL "invent" them from scratch (the "common view"), or are they artifacts of the training data?

The common view vs. the reality: RL rarely coins the "wait" marker; it enters during mid-training or SFT.

Methods

The researchers treated the training history of open models as a diachronic corpus—a collection of texts that changes over time. They analyzed three classes of model lineages:

  1. Fully open lineages: Tracking OLMo-3 from pre-training through mid-training to RL.
  2. Zero-RL runs: Training models like Qwen2.5 and Mistral from "marker-free" bases using only outcome rewards.
  3. Fine-tuned descendants: Models trained on reasoning traces (like R1-Distill).

To measure the evolution of "wait," they developed several label-free linguistic indices:

  • Frequency: How often the word appears.
  • Fixation: Whether it moves to the start of sentences.
  • Attrition: Whether the form "wears down" (e.g., dropping the comma in "Wait,").
  • Injected-Error Alarm Test: A clever probe where they intentionally insert an arithmetic error into a correct reasoning chain and see if the model triggers a "wait" marker.
Figure 2: Our methodology extracts a marker’s evolutionary history from its occurrences, one checkpoint at a time. The pipeline processes ❶lineages and sampled traces into ❷matched occurrences, from which we construct ❸label-free indices and validate the ❹injected-error alarm test, with statistics at every checkpoint (bottom).

Datasets

The study primarily used MATH-500 for evaluation, supplemented by AIME 2024/2025, AMC23, GSM8K, and MMLU-Pro. They also released a "provenance map" of 120 model endpoints and expert-consensus audits of reasoning traces.

Key Insights and Ideas

  • RL is not a linguist: In 17 out of 18 Zero-RL runs from bases that lacked the word "wait," the models achieved massive accuracy gains (median +42.9 points) while the word "wait" remained virtually absent. RL doesn't need the word to learn the logic.
  • The "Wait" comes from the data: In the OLMo-3 lineage, "wait" didn't appear during RL; it entered during the mid-training phase when reasoning-rich data was introduced.
  • Linguistic Attrition: When RL does have access to the marker, it "wears it down." In OLMo-3 RL-Zero, the canonical "Wait," form dropped from 89% to 42%, often becoming a repetitive "wait, wait" chain.
  • The Liberal Alarm: As RL progresses, the "wait" marker becomes a more sensitive but "liberal" alarm. It fires more reliably when there is an actual error, but it also starts firing on perfectly correct steps (false alarms).
OLMo-3 evolution: RL multiplies the frequency of "wait" but erodes its canonical form and sentence length.

Key Results

  • Zero-RL rarely coins markers: Models trained from scratch with RL on math problems almost never start saying "wait" if it wasn't in their base weights.
  • Data is the seed: 13 out of 14 models fine-tuned on reasoning traces adopted the marker from their training data.
  • Recipe matters: Different RL algorithms treat the marker differently. While standard RL-Zero recipes multiply the marker, a KL-free GRPO recipe actually erased a marker that had been seeded by fine-tuning.
  • Judge Unreliability: The authors found that "LLM-as-a-judge" metrics for reasoning can actually reverse human-labeled trends, over-calling revisions in base models and under-calling them in RL models.

Limitations and Future Work

The study is primarily observational and focused on English-language mathematics. While it provides a robust framework for tracking lexical markers, it doesn't fully capture "silent" reflection—where a model might be reconsidering its steps without explicitly writing a marker.

Future work could extend this "provenance tracking" to other domains (like coding) or other languages to see if the "inherited, not invented" rule holds globally across the LLM ecosystem. The authors also suggest that monitor-builders should calibrate their tools to specific model checkpoints rather than treating "wait" as a universal signal of doubt.

Summary updated

Abstract

Reinforcement learning (RL) is credited with teaching reasoning models to write {wait} before reconsidering a step, yet large RL accuracy gains do not require the word. This view can confuse what RL discovers with what a model inherits from its training data. We trace the word through open models' training histories, reading it with label-free indices of frequency, position, form and usage and an injected-error test, all computed from sampled outputs. We find that zero-RL almost never coins the marker: on bases that lack the word, RL raises MATH-500 accuracy substantially while {wait} stays near absent. It arrives with training data: mid-training in OLMo-3, and fine-tuning on reasoning traces installs it in most models whose base lacks it. Where the base already uses it, both released RL-Zero recipes (OLMo-3, MiMo) multiply it, whereas KL-free GRPO from the same base does not; OLMo-3's recipe wears down its form and makes it a more sensitive but more liberal alarm; RL after supervised fine-tuning barely changes it, and our GRPO recipe erases a marker seeded by fine-tuning. Finally, we show that marker frequency alone does not indicate RL-induced reasoning and caution against LLM-as-a-judge revision metrics, as automated judges can reverse human-labelled trends.
BibTeX
@misc{kou2026inherited,
  author = {Zhizhuo Kou and Sizai Hou and Long Chen and Jian Yang and Junyu Luo and Jingshu Peng and Wenkai Zhang and Sirui Han and Yike Guo},
  title = {{Inherited, Not Invented: Tracing the Provenance of Wait in Reasoning Models}},
  year = {2026},
  eprint = {2610.30882},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.30882},
}
Figure