When Safety Filters Shrink Specialized Models: Toxicity Gating in Recursive Domain Pre-Training

Language models are increasingly retrained on text written by earlier models, and developers usually pass that synthetic text through an automated toxicity scorer first. A toxicity scorer is a classifier that gives each passage a number between 0 and 1 for how hostile or offensive it is. In a general web corpus this looks harmless. In specialized corpora it may not be. Criminal case law describes assaults, civil-rights opinions quote slurs, and historical newspapers report wars. These are exactly the passages a toxicity scorer flags. If a filter removes them from each generation of synthetic data, and the next model learns only from what survived, the filter stops acting as a safety step and starts deciding, round after round, which parts of the domain the model gets to keep.

This note proposes to measure that effect directly and to test a simple correction. The core prediction is that a toxicity gate works as a topic-selective sampler. Its false positives cluster in particular legal and historical subtopics, especially those that mention identity groups. When the gate is applied recursively, those subtopics should shrink geometrically, faster than they would under random removal of the same amount of data. Meanwhile, subtler demographic bias, which the scorer does not detect, should stay put or become more concentrated. The correction we test is a stratum-preserving gate. It still removes toxic passages, but it removes them within each subtopic, and then reweights the kept data back to the subtopic frequencies of the original human corpus.

Core Claim

The prior work that motivates this proposal establishes three separate facts. First, filtering synthetic data by any reward accelerates the loss of diversity. Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences proves that retraining on curated samples acts like implicit preference optimization and drives variance to zero; on CIFAR-10, a confidence-based curator collapsed the model onto a single class. When Sample Selection Bias Precipitates Model Collapse shows that a narrow "quality" verifier performs worse than random selection, including for Llama-2-7B summarization on XLSum. Second, off-the-shelf toxicity scorers misfire on specialized text. Toxicity of the Commons documents Perspective API over-flagging neutral historical reporting of violence, and builds Celadon, a five-headed domain-aware classifier with weighted accuracy of about 0.73. Third, recursive training in specialized domains erases rare content and skews demographics. AI-generated data contamination erodes pathological variability and diagnostic reliability reports a 98.9% drop in unique radiology vocabulary by generation 4 and a rise in male representation from 53% to 70%.

No one has connected the three. The curation-collapse papers use abstract rewards on images or topic silos. The toxicity-curation paper filters once and never retrains recursively. The clinical paper never applies a toxicity gate. Our claim is specific and falsifiable. Let $p_s^{(t)}$ be the share of subtopic stratum $s$ (for example, violent-crime opinions or equal-protection cases) in the generation-$t$ training set. Let $r_s$ be the gate's retention rate within that stratum, and let $\bar r$ be its overall retention rate. If each generator roughly reproduces the distribution it was trained on, then

$$p_s^{(t)} \approx p_s^{(0)} \left(\frac{r_s}{\bar r}\right)^{t},$$

up to renormalization across strata. The form follows from multiplying the per-generation selection ratio $t$ times. A random gate with the same $\bar r$ keeps $r_s = \bar r$, so any stratum-level decay beyond sampling drift can be attributed to the gate's correlation with topic, not to the amount of data removed. We also expect the generator's own mode-seeking to add decay on top of this prediction, which we measure separately.

The second half of the claim concerns bias. Bias Amplification: Large Language Models as Increasingly Biased Media found that political bias grew across ten GPT-2 generations while text quality stayed stable, and that largely distinct neurons drive the two. We extend that decoupling to a new pair: explicit toxicity and stereotyped association. Our hypothesis is that a toxicity gate lowers explicit toxicity in the outputs but leaves the stereotyped associations untouched or stronger. FairBelief - Assessing Harmful Beliefs in Language Models shows such associations sit at the model's highest-likelihood completions, which a surface scorer rarely flags.

Research Program

The primary domain is United States case law, drawn from the Caselaw Access Project documents underlying CLERC: A Dataset for U. S. Legal Case Retrieval and Retrieval-Augmented Analysis Generation. This domain offers three things we need: violence-heavy subdomains, identity-heavy civil-rights subdomains, and a downstream retrieval task with published numbers. The replication domain is the historical newspaper portion of the Common Corpus, which comes with ToxicCommons labels and Celadon. We fix subtopic strata before any training. For law we use court and area-of-law metadata together with an identity-mention tagger. For newspapers we cluster documents by topic.

Each chain starts from a model continued-pretrained on the real domain corpus. This is domain-adaptive pre-training, meaning ordinary next-token training carried on over specialized unlabeled text. We use 1B to 1.5B causal models of the kind used in Recursive Training Loops in LLMs. At each generation, the current model generates a corpus of fixed size, a gate filters it, and the next model is continued-pretrained from the same base checkpoint on the kept text. Each chain runs eight generations. Three things vary across chains: which gate is used, the fraction of real data mixed in (0%, 25%, or 50%), and the retention rate, which is matched across gates at 70% and 90%.

The proposed stratum-preserving gate works in three steps. It scores passages with Celadon. It removes the top-scoring passages separately inside each stratum, so that $r_s = \bar r$ by construction. It then importance-weights the kept passages so that stratum shares match $p_s^{(0)}$ from the real corpus. The key design choice is that the anchor comes from real-corpus frequencies rather than from the base model's probabilities. That choice avoids anchoring to the base model's biased peaks, which is the failure we expect from Self-Verification Provably Prevents Model Collapse in Recursive Synthetic Training.

We track five families of measurements across generations:

  1. Stratum shares. We fit these against the geometric prediction above.
  2. Vocabulary retention within each stratum. This follows the clinical study's metric.
  3. Output toxicity. We measure it with both Celadon and Perspective, on domain prompts and on PolygloToxicityPrompts English prompts.
  4. Stereotype measures. These are HONEST scores and identity-conditioned toxicity, meaning the gap in completion toxicity between prompts that name different groups.
  5. Downstream utility. This is CLERC retrieval Recall@1000, measured after an identical retrieval fine-tune of each generation's model.

We also repeat the neuron-correlation analysis from the Bias Amplification paper. The question is whether, under domain pre-training, the neurons tied to toxicity and those tied to stereotype remain as separate as that paper found for bias and quality.

Baselines

The naive reference point is the ungated chain: no filter, 0% real data. It shows how much collapse and toxicity drift happens without any intervention. It is the condition closest to the human-out-of-the-loop regime in Recursive Training Loops in LLMs, which used no toxicity scoring.

The most important control is a random gate at matched retention. It discards exactly as many passages per generation as each toxicity gate, but chooses them uniformly at random. This isolates our key ingredient: topic-correlated selection versus simple data loss. Every toxicity gate is compared against the random gate at the same $\bar r$, the same generation budget, and the same training steps.

The standard practitioner baselines are two threshold filters. One drops passages with Perspective toxicity above 0.5. The other applies a Celadon severity cutoff, as in Toxicity of the Commons. The Celadon cutoff is the strongest existing toxicity-specific gate because it was built for exactly the historical-context blindness we worry about. The "detox over delete" variant from the same paper is also run. It rewrites flagged passages with an LLM instead of dropping them. It competes directly with our gate because it also tries to keep content while removing toxicity.

The strongest existing anti-collapse method is real-data mixing with quality filtering. The clinical contamination study reports that combining at least 25% real data with quality filtering improved diversity retention by 40%. The Bias Amplification paper reports that preservation held right-leaning output to about 41–42% at generation 10, but did not remove inherited skew. We run mixing at 25% and 50%, crossed with every gate, using the same real-data pool for all.

We also include self-verification anchored to the base model. It keeps only candidates within a log-probability margin of the best of $N$ samples under the initial model. On synthetic linear functions, it held error to 0.31 at round 3 versus 1.95 for naive recursion.

For downstream utility, the published reference is ft-LegalBERT DPR at 68.5% Recall@1000 on CLERC, reported in the CLERC paper. We treat our generation-0 model's recall as the ceiling. We found no reported result on Lacuna for toxicity-gated recursive legal pre-training, so all gate comparisons are new measurements.

To count as beating the baselines, the stratum-preserving gate must meet three conditions. Its per-stratum share decay must be no worse than the random gate's. Its explicit output toxicity must be no worse than the Celadon cutoff's. And by generation 8, its CLERC recall and per-stratum vocabulary retention must be significantly higher than every threshold and rewrite gate at matched real-data fraction.

Why not just…?

Why not just use the domain-aware classifier, Celadon, with a threshold? Celadon fixes much of Perspective's misreading of historical violence, but a better classifier does not remove the mechanism. Even correct flags are concentrated in particular strata. Recursion then compounds that concentration through the ratio $r_s/\bar r$, whatever the classifier's accuracy. The Celadon-threshold chain tests this directly. If its measured $r_s$ is close to $\bar r$ across strata and its decay matches random, our concern is unfounded for good classifiers, and we will report that.

Why not just mix in real data? This is the most serious threat. The clinical study's 40% diversity gain suggests mixing may absorb most of the gate-induced loss. Mixing does not, however, remove inherited skew, as the Bias Amplification paper showed. Real data is also finite, as projected in Will we run out of data?. The crossed design settles the question. If a Celadon gate plus 25% real data matches the stratum-preserving gate on stratum shares, recall, and bias, then the simpler recipe wins and our gate is only useful when real data is unavailable. We would say so.

Why not just rewrite toxic passages instead of deleting them? Rewriting keeps the topic but injects additional synthetic text into every generation. Strong Model Collapse shows that even a 0.1% synthetic share can stall learning. The rewrite chain measures whether rewriting keeps stratum shares but loses stratum vocabulary, for example by flattening the specific terms of a criminal opinion. That outcome would show a quieter version of the same collapse.

Why not just anchor to the base model's own confidence? Self-verification assumes the base model is well calibrated. Its most confident completions are also where FairBelief locates stereotypes. We expect this chain to keep fluency and CLERC recall while preserving or increasing identity-conditioned toxicity gaps. If it does not, that is itself a useful negative result.

Decisive Experiments

The first decisive check is the geometric prediction. From generation 1 we measure each gate's per-stratum retention ratio $r_s/\bar r$ and forecast stratum shares at generations 4 and 8. The claim survives only if the Perspective and Celadon threshold gates track their forecasts and decay significantly faster than the random gate in the flagged legal strata, meaning violent crime and equal protection. If the toxicity gates and the random gate are indistinguishable, the central claim is false.

The second check is the decoupling of toxicity and bias. The claim survives only if the gated chains lower Celadon- and Perspective-measured output toxicity relative to the ungated chain while HONEST scores and identity-conditioned toxicity gaps stay flat or grow. We will also test whether the neuron sets tied to toxicity and to stereotype overlap as little as the Bias Amplification paper found for bias and quality, or whether domain pre-training entangles them.

The third check is utility. We compare CLERC Recall@1000 across gates at generation 8 against the generation-0 model, with 68.5% for ft-LegalBERT DPR as an external reference. We then replicate the strata analysis on the Common Corpus newspapers, using ToxicCommons labels, to show the effect is not specific to law.

The outcome that would most change practice is this: explicit toxicity falls under standard gating while civil-rights content, domain vocabulary, and retrieval quality decay faster than under random filtering, and the stratum-preserving gate prevents that loss without giving back the toxicity reduction.