Spending Privacy Budget on Facts: Claim-Level Private Verification for Distilling from Synthetic Data

When a hospital or bank wants a small, deployable language model trained on its sensitive records, the usual route has two steps. First, produce a privacy-preserving synthetic corpus, meaning text generated under a formal guarantee (differential privacy, which bounds how much any single real record can change the output). Second, distill a compact student model on that corpus, so the student learns to imitate text a larger model produced. Both steps optimize for looking like the data: the generator matches the data distribution, and the student matches the teacher. Neither step checks whether a synthetic sentence states something the private data actually supports. A synthetic clinical note can be fluent and privately generated and still attribute a donor's characteristics to the recipient. That is exactly the entity-swapping failure Léo Milecki documents when LLMs write clinical notes from tables. A student distilled on such notes treats the error as ground truth and learns it.

This note proposes to spend part of the privacy budget directly on factual verification. Each synthetic document is broken into atomic claims. A small verifier model, distilled from a large one, runs inside the privacy boundary and asks each private record whether it entails or contradicts each claim. Only noisy, privacy-accounted support counts leave the boundary. Those counts are then used to filter and weight the synthetic corpus before the student is distilled. Put in two sentences: private data should be used not only to make synthetic text resemble it, but to vote on which synthetic claims are true of it. Because each record casts one bounded vote per claim, that vote can be released with a formal privacy guarantee.

Core Claim

Hallucinations in privately synthesized domain text split into two kinds. The first kind are general-knowledge errors, which any strong public model can catch. The second kind are dataset-specific relational errors: which attribute belongs to which entity, which finding co-occurs with which procedure, which tentative result is really only conditional. No public checker can catch these, because the truth lives only in the private records. The hypothesis is that the second kind dominates the factual errors a distilled student inherits in specialized domains.

It follows that a claim-level private support signal should reduce student hallucination beyond what the same budget buys when spent on a better generator. The same is predicted against filtering with a public judge. The support signal for claim $c$ is

$$\tilde{s}(c) = \sum_{i=1}^{n} v_i(c) + \mathcal{N}(0, \sigma^2),$$

where $n$ is the number of private records, $v_i(c) \in \{-1, 0, 1\}$ is the verifier's contradict, neutral, or entail judgment of claim $c$ against record $i$, and $\sigma$ is the Gaussian noise scale. Each record's contribution per verification round is capped at $m$ claims, so the per-record sensitivity is bounded. Standard Gaussian-mechanism composition then yields the privacy cost.

The novelty is not detecting hallucinations or generating private data. It is treating factual support as a privatized query answered by the private data, placed between synthesis and distillation.

Research Program

The pipeline has four stages.

The first stage is a privacy-preserving generator. Any standard private text synthesizer will do, and its budget $\varepsilon_{\text{gen}}$ is a controlled variable. The research file did not contain retrievable entries on specific private synthesis methods, so this note does not commit to one. The experiments will treat the generator as swappable.

The second stage extracts atomic claims from each synthetic document. Here I reuse a method from prior work. FENICE splits summaries into atomic content units and checks them with a DeBERTa-v3 natural language inference model plus coreference resolution. It also distilled its GPT-3.5 claim extractor into a compact T5-base model. Extracting and Encoding similarly has an LLM break radiology reports into atomic facts before training a small encoder. Both works assume the checked text and the reference text may be freely combined. Neither considers a reference corpus that cannot be released, which is the setting here.

The third stage is private verification. Distillation matters here for a practical reason, not only for model size. The number of verifier calls is roughly records × claims, and a closed-source or large teacher often cannot be run over sensitive records at all. So a compact entailment verifier is distilled from a large teacher on public data, following the weak-supervision recipe of OPDAI. In that work, small LoRA-tuned models trained on labels produced by larger models exceeded their larger supervisors at hallucination detection. The distilled verifier runs locally.

Each private record votes on a bounded set of candidate claims. The candidates are pre-selected by a public retrieval step keyed on claim entities, so each record is compared only to claims mentioning matching entity types. The noisy tallies $\tilde{s}(c)$ are released.

The fourth stage uses the signal. Claims with significantly negative $\tilde{s}$ are rewritten by the teacher or deleted. Documents are weighted by the fraction of supported claims. The student is then distilled on the reweighted corpus using standard sequence-level distillation from the autoregressive distillation literature, such as MiniLLM.

The analysis side asks three questions. How should a fixed total budget be split between $\varepsilon_{\text{gen}}$ and $\varepsilon_{\text{ver}}$? Which hallucination types are corrected? Does student hallucination track the uncorrected claim rate of its training corpus? That last relationship is the error-cascading concern the distillation literature raises but rarely measures.

A known tension deserves direct attention. Knowledge Overshadowing shows that strong parametric priors suppress weaker context-specific facts. That implies that fixing the corpus may not fully fix the student, whose pretrained priors may reassert themselves. The experiments therefore measure both corpus-level and student-level error.

Baselines

The naive reference point is distilling the student on the unfiltered private synthetic corpus, with the entire budget spent on generation. This is the default pipeline, and it measures how much hallucination the student inherits when nothing intervenes.

The strongest simple competitor is public-judge filtering. The same claims are extracted from the same synthetic corpus and scored by a strong public model, or by the FENICE-style entailment checker against public domain references. Failing claims are removed and the whole privacy budget goes to generation. This competitor costs nothing in privacy and catches general-knowledge errors. It is run with the same claim extractor, the same filtering thresholds tuned on the same held-out development set, and the same student training compute.

The strongest alternative use of the private data is private fine-tuning of the student directly on the records at the same total $\varepsilon$, with no synthesis. It gets an equal hyperparameter search budget.

The key ablation isolates the private-support ingredient. It keeps the full pipeline but replaces $\tilde{s}(c)$ with noise-only scores that carry the same filtering rate. This separates "filtering helps because it removes text" from "filtering helps because the private votes carry information."

A second ablation varies the budget split at a fixed total $\varepsilon$. On the Lacuna search, no reported result was found for any private-synthesis-then-distillation pipeline on clinical hallucination. There was also no reported number for FENICE-style filtering in a private setting. So the baselines must be run from scratch.

Beating the public-judge baseline means a statistically significant reduction in the student's rate of contradicted atomic claims on held-out real records, at equal total privacy budget, with no loss in task utility. The reduction should be concentrated in relational errors such as entity swaps, as the Core Claim predicts.

Why not just…?

Why not just filter synthetic data with a strong public judge and spend nothing on verification? This is the real threat. For general-knowledge claims it probably matches the proposal at zero privacy cost. The argument for private voting rests entirely on relational, dataset-specific errors, where a public judge has no ground truth: a public model cannot know which attributes belong to the recipient in this hospital's tables. The experiment settles the question by stratifying corrected claims by type. If the public judge closes most of the gap on relational claims too, the proposal is wrong and should be dropped.

Why not just fine-tune the student on the private data with differential privacy and skip synthesis? This removes a whole stage and its error source. However, private fine-tuning gives no inspectable intermediate corpus, which matters when the student must be audited. It also offers no handle on hallucination: the student still extrapolates from its priors when records are ambiguous, as Jingwei Ni et al. and David Kartchner et al. describe. The equal-budget comparison will show whether it nonetheless hallucinates less. If it does, the synthesis route needs a different justification.

Why not just spend the verification budget on a better generator? If generator quality at higher $\varepsilon$ reduces relational errors as fast as verification does, the extra machinery is unnecessary. The budget-split sweep answers this directly. The proposal predicts an interior optimum where some $\varepsilon_{\text{ver}} > 0$ beats $\varepsilon_{\text{ver}} = 0$ at every total budget tested.

Why not rerank candidates at inference time instead of cleaning the training data? Co-opNet shows that the most factual candidate is often not the most probable one. TRAC adds conformal guarantees over answer sets. Both add multi-stage inference cost, which defeats the purpose of a compact student. They also need access to references at deployment time, which is exactly what privacy forbids. Inference-time reranking with a public verifier will be reported as a complementary comparison, not a substitute.

Proposed Experiments

The primary setting is structured-to-text clinical note generation of the kind studied by Milecki, with a radiology-report setting following Extracting and Encoding as a second domain. The records are held out as the private set, and real held-out records serve as evaluation references.

The verifier is validated on the SemEval-2024 Task 6 hallucination detection data used by OPDAI. This confirms that distillation did not degrade detection before the verifier is trusted inside the boundary. No reported score for that comparison was compiled in the file, so both teacher and distilled verifier are measured here.

Student outputs are scored three ways. The main metric is FENICE-style atomic-claim contradiction rate against the true source record. Entity-swap rate is scored by exact attribute-to-entity matching, which is possible because the source is a table. Task utility is measured by downstream accuracy on the structured fields.

Total privacy budgets are swept over a small grid, with three seeds each.

The decisive checks are three. First, at equal total $\varepsilon$, the private-support pipeline must beat public-judge filtering on entity-swap rate while tying on general-knowledge claims. Second, the noise-only ablation must perform like no filtering. Third, student contradiction rate should rise roughly monotonically with the uncorrected contradicted-claim rate of its training corpus. Such a relationship would give the first direct measurement of hallucination cascading through distillation, and its absence would suggest overshadowing by pretrained priors dominates.

If the first check fails, private verification is not worth its budget and the note's claim is falsified.