When a language model writes a long answer, one practical way to remove hallucinations is to split the answer into atomic claims (short, self-contained statements that are each true or false on their own), score each claim with a verifier, and delete the claims that fall below a cutoff. The cutoff is what carries the safety promise. Conformal methods, which set a cutoff from a held-out labeled sample so that a target error rate holds on average, give that promise, but only when the labeled sample comes from the same domain as deployment. A Wikipedia-calibrated filter moved to medicine loses 4–6 points of false-discovery control. The anomaly-detection literature has faced the same problem for years: how to pick an alarm threshold when real anomalies are scarce. Its main answer is to inject synthetic anomalies and tune the threshold against them.
This note proposes to bring that answer to claim filtering, with one change that we think decides whether it works. Synthetic hallucinations should not replace real labels. They should be reweighted by a very small set of real unsupported claims from the target domain, so that the synthetic errors resemble the real ones in how the verifier scores them. Atomic decomposition is what makes this possible. Because each perturbation edits exactly one atomic fact, we can sort synthetic errors into interpretable families (entity swap, number change, negation, relation shift, plausible unsupported addition) and estimate how common each family is among real errors. The claim we will test is this: with 10–25 in-domain labels plus a large reweighted synthetic pool, the cutoff controls the false-discovery rate (the fraction of retained claims that are actually unsupported) as reliably as conformal calibration on 200–500 in-domain labels, and more reliably than either the same few labels alone or a large out-of-domain calibration set.
Where this sits
The closest work splits into two camps that have not been combined. CiteGuard splits retrieval-augmented answers into claims, scores them with a DeBERTa NLI cross-encoder, and converts the scores into conformal p-values. To build those p-values it needs 200–500 labeled unsupported claims, and it then applies Benjamini–Hochberg (or Benjamini–Yekutieli under correlated errors) at budget α. Its cross-domain failure is the gap that motivates this proposal. Language Models with Conformal Factuality Guarantees gets by with 25–50 annotated examples, but its guarantee assumes the calibration and test prompts are exchangeable, and it holds only on average across queries. With so few points, the realized error rate of any single calibration draw varies widely. We reuse both papers' pruning setup. We change where the calibration information comes from.
On the synthetic side, Learning Ensembles of Anomaly Detectors on Synthetic Data tunes an ensemble threshold to maximize F1 on injected spikes and drifts, reaching about 0.77 F1 where single detectors reach 0.30–0.50. The same file also records that this approach succeeds only when the injected anomalies match real ones. Enhancing Hallucination Detection through Perturbation-Based Synthetic Data Generation perturbs whole responses to train a T5 verifier, reaching 0.762 F1 on OpenDialKG-Eval against 0.608 for zero-shot GPT-4. It works at the response level, though, and keeps a default 0.5 cutoff. We supply the method neither paper has: perturbations made at the atomic level, used to set the threshold rather than to train the verifier, and corrected for mismatch with real errors.
Two papers constrain the design. A Closer Look at Claim Decomposition shows that the choice of decomposition prompt shifts downstream factuality scores substantially. Its Russellian/neo-Davidsonian prompt reaches a DecompScore of 42.3, against roughly 32 for the FActScore prompt. If decomposition is sloppy, a "single-fact perturbation" may in fact edit several facts, so we adopt its decomposition and measure decomposition quality as a covariate. CGRiC warns that claims missed during extraction cannot be certified at all. Our method inherits this limitation, and our guarantee is stated relative to the claims that were extracted and to the verifier, as in CiteGuard.
Core Claim
Real hallucinations are not a uniform mix of edit types. Retrieval-grounded medical answers, for instance, may fail mostly through unsupported additions, while biographies fail through wrong dates and entities. Verifiers catch contradictions far more easily than unverifiable additions. An unweighted synthetic pool is therefore likely to be dominated by easy contradictions, and a threshold tuned on it will be too lenient. If instead we estimate the mixture weights $w_f$ over perturbation families $f$ so that the weighted synthetic score distribution matches the scores of the few real unsupported claims, we get a calibration null that has low variance, because it contains thousands of points, and low bias, because it is anchored to real errors. Concretely, we choose $w$ on the probability simplex to minimize the distance between the weighted synthetic score distribution $\sum_f w_f \hat{F}_f(s)$ and the empirical distribution $\hat{F}_{\text{real}}(s)$ of verifier scores $s$ on the $n$ real unsupported claims. Here $\hat{F}_f$ is the empirical score distribution of family $f$. The cutoff is then read off the weighted pool and plugged into CiteGuard's BH/BY procedure.
This construction does not inherit conformal finite-sample validity, and we will not claim that it does. To give a checkable promise, we add a second step. The few real labels are split: one part fits the weights, and the other is used to run a one-sided binomial test that the chosen cutoff's error rate on real claims stays below α. If the test fails, we fall back to the next more conservative cutoff.
Research Program
The first component is the perturbation generator. Supported claims, obtained by decomposing reference-consistent answers with the $D_{R\text{-}ND}$ prompt, are each edited by an LLM within one of five families. Each edit is checked by the verifier's own NLI model for single-fact change against the original, and edits that change more than one fact are discarded. The second component is the weight estimator together with the held-out binomial check. The third is an analysis component that relates calibration error to two quantities: the distance between the synthetic and real score distributions, and the DecompScore of the decomposition used. This analysis lets us say when the method should be trusted.
We deliberately keep CiteGuard's verifier, its pruning rule, and its BH/BY switch fixed. The only thing that varies is where the calibration null comes from.
Proposed Experiments
The main domain-shift setting calibrates and develops on FActScore biographies and Natural Questions, and deploys on PubMed summaries and MedQA-Cite. The medical sets mirror CiteGuard's Wikipedia-to-medicine failure. We also run FEVER as a sanity check on in-domain verification. Ground truth for evaluating claims comes from FActScore's evaluator on biographies, which its paper reports is within 2% of human experts. For the medical sets we annotate roughly 1,500 test claims by hand, because no automated reference there has been validated. We vary the number of in-domain real labels $n$ over 0, 10, 25, 50, and 200, and repeat every condition over 200 random calibration draws.
The metrics are realized FDR at α ∈ {0.05, 0.10, 0.20}, the fraction of calibration draws that violate α, and the fraction of supported claims retained. We report retention because a cutoff can always reach a low FDR by deleting almost everything.
Baselines
The naive reference point is the verifier's default 0.5 cutoff with no calibration, which is the operating point used in the perturbation-trained detector paper. Alongside it, we include the static $k\sigma$ rule familiar from anomaly detection. CiteGuard reports that uncalibrated filtering on Natural Questions leaves the FDR at 30.6%, which shows how far these reference points can drift.
The strongest existing method is CiteGuard's conformal BH/BY calibrated on 200–500 in-domain labeled unsupported claims. On Natural Questions with α = 0.10, it reports reducing FDR to 9.4% while keeping 86–92% of supported claims. We rerun it with the same verifier and the same pruning code, using 200 in-domain labels drawn from the medical annotation pool. This serves as the target our method should match. Under the same few-label budget we also run Conformal Factuality-style split conformal calibration, which reports factuality on Natural Questions rising from 78% to 93% after removing about 25% of claims with 25–50 calibration samples, together with CiteGuard calibrated on the out-of-domain Wikipedia/NQ set. These two runs are the competitors our method must beat outright.
Two ablations isolate our key ingredients. The first is the unweighted synthetic pool, which is a direct transfer of the synthetic-anomaly approach from time series. The second applies the same reweighting to perturbations made at the sentence level rather than the atomic level.
Every method sees the same test claims and the same verifier, and none may tune hyperparameters on the test annotations. Our method counts as better if, at $n$ = 25, its realized FDR stays within one point of α in at least 90% of calibration draws, and its retention is within three points of 200-label CiteGuard while being better than both few-label conformal calibration and out-of-domain calibration.
Why not just…?
Why not just label 25–50 in-domain claims and run ordinary split conformal? This is the real threat, because labels at that scale are cheap and the guarantee is exact. The weakness is variance across calibration draws. With $n$ points the conformal quantile moves in steps of $1/(n+1)$, so a filter that is correct on average can be badly lenient on the draw a practitioner actually gets. If our sweep over $n$ shows that the violation fraction of plain conformal at $n$ = 25 is already as low as ours, the reweighting adds nothing, and we will report that outcome.
Why not just reuse a large out-of-domain calibration set? CiteGuard itself measures a 4–6 point loss of FDR control from Wikipedia to medicine. Out-of-domain calibration appears in our design as a baseline, not as a strawman.
Why not just use synthetic perturbations without reweighting, as the time-series work does? That work notes that its results depend on synthetic and real anomalies matching. The unweighted ablation tests directly whether the family mismatch between easy contradictions and hard unsupported additions causes lenient cutoffs.
Why not just train a better verifier on synthetic errors and keep a fixed cutoff? A better ranking does not fix the operating point. Tabular detectors that exceed 89% AUROC (Tuning-Free One-Class Discriminant Learning) still need an external threshold. We will include the synthetic-trained verifier at 0.5 to show how far its FDR drifts from α across domains.
Decisive Checks
The idea holds if three things are true at $n$ = 25 on the medical sets. First, the reweighted cutoff's violation rate is clearly below that of both plain conformal and out-of-domain calibration. Second, the unweighted synthetic pool is measurably too lenient. Third, the advantage of atomic-level over sentence-level perturbation grows with DecompScore, which would confirm that clean atomic decomposition is what makes synthetic errors usable for calibration. If plain few-label conformal matches us on violation rate and retention, we drop the claim.
References
- CiteGuard: Conformal False-Discovery Control for Faithful Retrieval-Augmented Generation
- Language Models with Conformal Factuality Guarantees
- Learning Ensembles of Anomaly Detectors on Synthetic Data
- Enhancing Hallucination Detection through Perturbation-Based Synthetic Data Generation in System Responses
- A Closer Look at Claim Decomposition
- CGRiC: Compositional Risk Certification for Structured LLM Outputs
- Tuning-Free One-Class Discriminant Learning for Tabular Anomaly Detection