Hard or Unanswerable? Identifiability-Gated Curation of Reasoning Data

Training sets for reasoning models are increasingly built by sampling many solutions and keeping the ones that reach the labeled answer. Recent curation methods deliberately spend more of that sampling budget on questions a reference model usually gets wrong. That is sound when a question is hard because it needs long computation. It goes wrong when a question is hard because the input does not contain enough information to determine the answer. For those questions, the "correct" trace is a lucky guess dressed up as a derivation, and training on it teaches the model to sound confident where it should not be. Phylogenetics gives a clean place to study this. When species split faster than their DNA mutates, the alignment simply does not record which lineage branched first. That is the field's well-documented resolution limit. In simulation, how much information each instance carries can be computed exactly.

We propose to use simulated tree inference and kinship inference as measurement instruments. Both are reasoning tasks where the best achievable accuracy is computable for every single instance. We will use them to test one claim: difficulty-weighted curation helps only on instances that are hard but identifiable, and it actively harms on instances near the information limit. We then test a cheap, oracle-free gate borrowed directly from phylogenetics, the bootstrap support value. The gate decides which hard instances deserve extra sampling budget and which should be relabeled as unresolved or dropped.

Core Claim

A model's fail rate mixes two quantities that curation should treat oppositely. One is computational difficulty: the evidence determines the answer, but finding it takes work. The other is information deficit: the evidence does not determine the answer. For an instance $x$ with candidate answers $\tau$ (for quartets of species, the three possible unrooted tree shapes), define its identifiability as

$$\iota(x) = \max_{\tau} P(\tau \mid x),$$

the posterior probability of the most likely answer under the true generating model. $\iota(x)$ is also the best accuracy any solver can achieve on $x$. On instances with $\iota$ near $1/3$, a trace that "derives" the simulated true topology carries no transferable signal. We predict three things. First, difficulty-aware rejection sampling in the style of DART-Math over-allocates samples to these instances. Second, doing so raises the model's rate of confident wrong answers on held-out data. Third, gating the difficulty allocation by an estimate of $\iota$ recovers the benefit of hard-example oversampling without that harm.

The metric that matters is excess error over the Bayes limit, not raw accuracy:

$$\mathcal{E} = \mathbb{E}_x\big[\iota(x) - \mathrm{acc}_\theta(x)\big],$$

where $\mathrm{acc}_\theta(x)$ is the model's probability of answering $x$ correctly. Alongside it we track the false-resolution rate: the fraction of instances with $\iota(x) < 0.5$ on which the model commits to a single answer instead of reporting "unresolved."

Why These Two Domains, and What Prior Work Establishes

The biological literature documents the resolution limit as a practical failure. Conserved chloroplast regions could not resolve fine branches in Elatostema (Hadiah et al.). Plastid barcodes lacked variation in rapidly diversifying Crawfurdia, where nuclear ITS succeeded (Zhang et al.). Two Zingiber species share near-identical barcodes and needed metabolite profiles to tell apart (Wang et al.). The chloroplast region trnT-trnL could not separate 430 millet accessions (Yu et al.). These papers motivate the proposal: they show that "no signal in this marker" is a common, real condition, not a corner case.

Forensic kinship has the same structure. Rohlfs et al. show that a 13-locus short tandem repeat (STR) protocol flags half-siblings as full siblings up to 42% of the time. That is a Bayes-limited confusion, not a solver failure, and it can be simulated with exact likelihood ratios.

On the ML side, two strands frame the tension we resolve. DART-Math allocates rollouts by a reference model's fail rate and outperforms datasets four times larger. OpenThoughts finds the opposite pressure: restricting to the top one or two sources, and favoring concise traces, beats mixing many sources. Neither separates the two reasons an item can be hard. We take DART's allocation as the method we modify and OpenThoughts' filtering as a competitor.

Mechanistic selectors also serve as competitors rather than ingredients. AIR scores traces by the loss change when the attention heads responsible for copying and long-range lookup are masked. TEMP scores the loss spike on the first 100 tokens under random weight noise. Both measure the model's reaction to a trace, and nothing in either measures whether the input could support the answer. DC-W2S is the closest cousin. It filters noisy step-level labels for biological process reward models (models that score each reasoning step) by agreement among weak judges, and finds that 100k curated instances beat 351k uncurated ones. DC-W2S, however, targets noise in the supervision. We target instances whose ground truth is correct but unrecoverable, a case where judges can agree perfectly and still be teaching a guess.

The bridge papers constrain the design. PHYLA shows that scaled protein language models do not acquire tree reasoning and often fail to beat Hamming distance. Classical pipelines still win on absolute accuracy, and the paper explicitly leaves micro-radiations unaddressed. We therefore do not claim an LLM will out-infer maximum likelihood. Likelihood is our oracle, and the LLM is the subject of the curation study.

Research Program

We first build a generator of reasoning instances with exact identifiability. For quartets, we simulate four aligned sequences under a standard substitution model, with internal branch length and alignment length as dials. We compute $\iota(x)$ by integrating the likelihood of each of the three topologies over branch lengths under the known model. We sweep from deep, well-separated splits down to near-zero internal branches, which mimic rapid radiation, so the pool spans $\iota$ from about $1/3$ to about $1$. To create genuine computational difficulty independent of information, we add instances with uneven rates across lineages. These require reasoning past the naive "count shared mutations" heuristic, which fails in exactly the long-branch cases. The kinship generator simulates 13-locus STR profiles for full siblings, half-siblings, cousins, and unrelated pairs, using a public allele-frequency table, and computes exact posteriors over relationship.

Next we produce reasoning traces two ways. The first follows Code Execution as Grounded Supervision: we execute a parsimony or likelihood program and render its intermediate quantities as natural-language steps. The second distills from a large reasoning teacher, sampling up to 16 traces per instance, which is the scaling axis OpenThoughts found effective. Traces are accepted by final-answer match, as in STaR-style rejection sampling.

The proposed curation rule is identifiability-gated difficulty allocation. We estimate $\hat\iota(x)$ without the generating model by phylogenetic bootstrap: resample alignment columns, or STR loci, rerun a cheap classical solver, and take the support of the majority answer. Instances with $\hat\iota$ above a threshold receive DART-style budgets proportional to fail rate. Instances below it are handled in one of three ways, each tested as an arm: dropped; relabeled with the target "unresolved; candidates A or B"; or given a trace that ends with the bootstrap support as a stated confidence. The oracle variant uses true $\iota$ and bounds what the gate can achieve.

Finally we check scaling. For each curation rule, we train at several data budgets and fit how $\mathcal{E}$ decreases with budget, separately on identifiable and information-limited held-out instances. On the information-limited slice we expect training data to buy nothing, because $\iota$ caps accuracy there. That flat curve is itself the diagnostic: any rule that raises accuracy above $\iota$ in training but not at test is memorizing simulation noise.

Baselines

All arms share the same student model (a 7B instruction model), the same trace pool, the same number of training tokens, and the same tuning budget of three learning rates by two epoch counts, selected on a held-out validation split.

The naive reference is random selection from the accepted-trace pool at matched size. Training on the full pool is a second naive reference.

The strongest existing allocation method is DART's difficulty-aware rejection sampling in both its Uniform and Prop2Diff variants. Fail rates come from the same reference model used by our gate. On MATH, DART-Math-Llama3-8B reports 46.6% accuracy (DART-Math). We found no reported result on Lacuna for DART or any curation method on phylogenetic or kinship reasoning tasks, so all in-domain numbers will be produced here.

The strongest selection method is AIR-Samp. On s1K it reports 54.78% average accuracy against 47.85% random, 52.50% length, and 49.89% entropy (AIR). We run AIR with a 7B reference model, since the paper reports smaller references score worse. We also run TEMP, OpenThoughts-style conciseness filtering, and the model's own self-consistency, meaning answer agreement across its 16 samples, as a gate.

The key ablation swaps our input-resampling gate for self-consistency while holding the allocation rule fixed. This isolates whether evidence-side uncertainty matters beyond model-side uncertainty.

Beating DART means a lower false-resolution rate on $\iota < 0.5$ instances with no loss in accuracy on $\iota > 0.9$ instances. Beating AIR means lower excess error $\mathcal{E}$ at equal budget, with non-overlapping confidence intervals across three seeds.

Why not just…?

Why not just run maximum likelihood and skip the language model? For producing trees, one should. Our oracle is likelihood, and PHYLA's own results concede classical pipelines are more accurate (PHYLA). The question here is not phylogenetic inference. It is how curation choices shape what a reasoning model learns about the limits of its evidence. These domains are the rare settings where that can be scored per instance.

Why not just gate by the model's own self-consistency? This is the real threat, because it is free and needs no domain solver. Our expectation is that it fails to separate the two kinds of difficulty. A weak model is inconsistent both on long-branch instances it could learn to solve and on unresolvable ones, so gating by self-consistency would discard exactly the hard, learnable examples DART was designed to keep. The ablation above settles this directly. If self-consistency gating matches bootstrap gating on both slices, we will report that the cheaper gate suffices and the contribution narrows to the diagnosis.

Why not just train everything with soft labels or label smoothing? Uniform smoothing lowers confidence everywhere, including on instances the evidence fully determines. It should raise excess error on the identifiable slice. We include a smoothing arm tuned to match our arm's calibration and compare their accuracy on $\iota > 0.9$ instances.

Why not just apply DC-W2S-style judge consensus? Consensus catches wrong labels. Here the labels are correct by construction, since simulation knows the true tree, so consensus has nothing to catch. Running it as an additional baseline tests this prediction.

Proposed Experiments and Decisive Checks

The first decisive check is the harm prediction. On the simulated quartet and kinship pools, DART-Prop2Diff should show a higher false-resolution rate than random selection at matched budget. If it does not, information-limited instances are not hurting, and the proposal's premise fails.

The second check is gate fidelity. Bootstrap support should rank-correlate with true $\iota$. The gated arm should close most of the gap between DART and the oracle-gated arm.

The third check is transfer to real data. On TreeBase and TreeFam alignments we extract quartets and treat nodes that the published trees support strongly as identifiable. Weakly supported nodes are treated as near-limit, in the spirit of the radiations in Elatostema and Crawfurdia. We measure accuracy on the first group and the rate of confident calls on the second.

The fourth check is cost to general reasoning. We fine-tune on a mixture of curated phylogenetic data and a fixed math set, then evaluate MATH500 and GPQA Diamond. This verifies that the gate does not degrade general reasoning. OpenThinker3-7B's reported 53.7% on GPQA Diamond (OpenThoughts) is the reference ceiling for 7B models.

A result where gating helps in simulation but not on real data would point to model misspecification in the bootstrap gate. That would be a useful, publishable negative result in its own right.