Training AI-Text Detectors on Private Human Writing

Detectors of machine-generated text fail worst on the people they are most often used to judge. A detector tuned on one kind of writing and applied to another can call most human texts machine-written: MAGE: Machine-generated Text Detection in the Wild reports that unadjusted supervised detectors labeled over 60% of human texts as AI on unseen domains. Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts notes that false positives are concentrated on non-native English essays. The fix is to fit the detector to real human writing from the target population, such as student essays, applicant statements, or support tickets. That writing is sensitive, and a fine-tuned classifier can memorize and leak it. Differential privacy (DP), a guarantee that the trained model barely changes when any single person's record is added or removed, is the standard remedy. What DP costs on this particular task has not been measured.

This task has a structural feature that general private-learning work does not use. Only one class is private. The human texts are sensitive, but machine texts from the same prompts can be generated in unlimited quantity and carry no privacy cost. We propose one-sided private training. Noise and clipping are applied only to gradients from the private human examples. Machine examples enter training at no privacy cost. Classification is posed as a fill-in-the-blank question to the model's own pretrained word-prediction head rather than through a newly initialized classifier. We then ask whether the finding that larger models lose less accuracy under DP, established for sentiment and entailment, holds for a task whose signal is statistical texture rather than meaning.

Core Claim

We make two linked claims.

The first is practical. A cloze-formulated detector trained with one-sided DP-SGD reaches meaningfully higher true-positive rate at a 1% false-positive rate on a sensitive human population than two alternatives at matched privacy budgets. The first alternative is a zero-shot statistical detector whose threshold is set privately. The second is a detector trained only on public data. The margin should be largest under domain shift and for writers underrepresented in public corpora.

The second claim is scientific. Privacy-utility scaling for detection behaves differently from the classification tasks where it was measured. The explanation offered for "bigger is better under DP" is that downstream tasks live on a low-dimensional slice of pretrained semantic features. This explanation is given in Differentially Private Fine-tuning of Language Models and in Large Language Models Can Be Strong Differentially Private Learners. Detection instead depends on token-probability smoothness and reduced geometric spread, the signals exploited by Training-free LLM-generated Text Detection by Mining Token Probability Sequences and the intrinsic-dimension paper. If those signals are not on the semantic slice, DP noise should hurt detection more, and the benefit of scale should be flatter.

We treat either outcome as informative. Steep scaling would mean detection is "semantic enough" for DP fine-tuning. Flat scaling would mean private detection needs a different recipe, for example leaning harder on the public machine class.

Where This Sits in Prior Work

The method comes mostly from two papers. Large Language Models Can Be Strong Differentially Private Learners showed that DP-SGD fine-tuning works well when three things are combined. The first is large batches and high learning rates. The second is Ghost Clipping, which computes each example's gradient norm without storing each example's gradient. The third is a text-infilling (cloze) formulation that avoids training a random classification head under noise. That work evaluated GLUE and generation tasks up to 774M parameters and did not consider detection. Improving and Simplifying Pattern Exploiting Training supplies the loss design. Its ADAPET method adds a decoupled binary loss over label words and a label-conditioning objective, and matched GPT-3 on SuperGLUE with 32 examples. Its mechanism, keeping the pretrained head in place so scarce or noisy gradients do not overwrite representations, is the same one that should matter under DP. ADAPET has never been run privately.

Three papers constrain the design. Scaling Laws for Differentially Private Language Models shows that when training from scratch under DP, the compute-optimal model is 10x to 50x smaller than without DP. It also shows that the noise-to-batch ratio is the right control variable. So we cannot assume that bigger is better, and we will report curves against that ratio. Flocks of Stochastic Parrots shows that prompts built from private examples leak membership, with membership-inference AUC of 0.84 on DBPedia. That motivates a leakage audit here. Its PromptDPSGD and PromptPATE methods are alternative private adapters, and we compare against the cheaper of the two. Prompt-and-Align applied cloze prompting to few-shot fake-news detection. That task concerns human misinformation, not authorship, and the work involves no privacy.

On the detection side, zero-shot scores are the main threat because they use no private training data. These are DetectLLM's log-rank ratios (DetectLLM), Lastde's multiscale entropy of token probabilities, and persistent-homology dimension. MAGE's Boundary Adjustment shows that a handful of in-domain samples can recalibrate a detector's threshold. MGTBench supplies the essay benchmark and the non-private supervised ceiling. Machine-Generated Text Localization shows that single sentences carry little signal, so we evaluate short-text regimes separately rather than assume transfer.

Research Program

The detector. An encoder with a word-prediction head reads the text followed by a template such as "This text was written by a [MASK]." The two label words are "person" and "machine." For decoder models, the same question is asked as the next token.

Training loss. We train with ADAPET's decoupled binary loss. Label conditioning is included as an ablation, because its extra reconstruction targets consume gradient norm under clipping and may not pay for themselves under DP.

The one-sided mechanism. Each batch mixes a Poisson-sampled set of private human texts with a batch of public machine texts. Private per-example gradients are clipped to norm $C$ and receive Gaussian noise. The public gradients are clipped to the same norm for scale stability but receive no noise. Privacy is accounted only over the private human pool, with sampling rate $q = |B_{\text{priv}}|/N_{\text{priv}}$. The public data is fixed and independent of the private records, so the guarantee is standard DP with respect to each human author. Mixing public data into private training is not new as a general idea. What is specific here is that the label structure makes one entire class public, so the ratio of public to private data is a free and very large knob.

Scaling sweeps. We sweep encoder sizes from base to large and GPT-2 sizes up to 1.5B. We also run a 7B decoder with DP-LoRA to cross the 774M scale boundary noted in the research file. Privacy budgets are $\epsilon \in \{0.5, 1, 3, 8, \infty\}$ with fixed $\delta$, and the noise-to-batch ratio is recorded for every run. We fit TPR at 1% FPR as a function of model size and $\epsilon$. The identical pipeline, including the same cloze machinery, DP optimizer and tuning budget, is also run on SST-2. This isolates how steeply detection scales relative to a semantic task.

Baselines

Naive reference. A public-only cloze detector. It is trained non-privately on human text from public corpora (WritingPrompts and WikiText human sides) plus machine text, and is applied to the sensitive population with a default threshold. This is what a practitioner who refuses to touch private data would ship. MAGE predicts it fails under domain shift.

Strongest zero-shot competitor. Lastde++ and the persistent-homology detector, with the decision threshold set on the private human pool by a DP quantile estimate. This is effectively a private version of MAGE's Boundary Adjustment. It is one-sided and cheap, and it spends privacy only on one scalar. For reference, Lastde reports average AUROC of about 95.89% in white-box testing and beats Fast-DetectGPT by 5.8 AUROC points black-box (Training-free LLM-generated Text Detection by Mining Token Probability Sequences). Persistent-homology dimension reports 40% accuracy at 1% FPR on GPT-3.5 (Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts). We run both with the same proxy scoring models across all conditions, at every $\epsilon$ in the grid.

Strongest supervised reference. A fine-tuned BERT classifier trained non-privately with full access to the private essays. MGTBench reports its F1 as 0.99 on Essay against ChatGPT (MGTBench). This is the ceiling DP is paying against, not a competitor.

Strongest private-fine-tuning competitor. DP-LoRA with a standard linear head, following Differentially Private Fine-tuning of Language Models. That paper reports 87.8% MNLI at $\epsilon=6.7$ versus 90.2% non-private. We also run PromptDPSGD with soft prompts. We found no reported result on Lacuna for any DP-trained machine-generated-text detector, so all private detector numbers will be new.

Fairness controls. All trained methods get the same private pool, the same public machine pool, the same hyperparameter-search budget with tuning privacy accounted or done on a disjoint public proxy, and matched compute.

Ablations. Two ablations isolate the proposal's ingredients. First, cloze versus linear head, holding the one-sided DP recipe fixed. Second, one-sided versus two-sided DP, treating machine texts as private too, holding cloze fixed. We also vary the public-to-private ratio.

What counts as beating a baseline. The proposal wins if, at $\epsilon \le 3$, it exceeds the private-threshold zero-shot baseline in TPR at 1% FPR on the held-out sensitive population by a margin larger than seed-to-seed variation. It must also cut the public-only baseline's human false-positive rate on shifted domains.

Why not just…?

Why not just use a zero-shot detector, which needs no private training data at all? This is the real threat. With a privately set threshold, it may already capture most of the achievable gain, since MAGE suggests calibration rather than representation is the main failure under shift. Our design includes exactly this baseline at every $\epsilon$. If it matches the trained detector, the honest conclusion is that private data should be spent on calibration, not training, and we will report that. We expect training to help in two regimes. The first is paraphrased machine text, where MAGE shows AUROC dropping from 0.94 to 0.75 and statistical signatures are partly erased. The second is short texts, where Lastde fails below 30 words. But this is an expectation to test, not a result.

Why not just train on public human text and skip privacy? Public corpora underrepresent the populations that suffer false positives, such as non-native writers, and MAGE's result that over 60% of human texts were misclassified under domain shift is evidence that this gap matters. The public-only baseline measures it directly on held-out essays from writers absent from public data.

Why not generate DP synthetic human essays and train non-privately on those? Synthetic Text Generation with Differential Privacy makes this a credible recipe. There is a specific worry for detection, however. Synthetic "human" text is itself machine-generated, so it may carry exactly the smooth-probability signature the detector must learn to reject, blurring the boundary. We include this pipeline at matched $\epsilon$. If it wins, that is an important finding in itself.

Why not DP-LoRA with an ordinary classifier head? This is the cloze ablation. The evidence that cloze helps under noise comes from classification tasks in Li et al. and in the few-shot ADAPET results. Whether it helps when the signal is statistical rather than semantic is precisely what the ablation tests.

Proposed Experiments

Data. The main testbed is MGTBench's Essay set. Human essays form the private pool. Machine essays generated from matched prompts by several models form the public pool. TOEFL-style non-native essays, as used in the persistent-homology work, serve as a held-out sensitive population. WritingPrompts and WikiText provide public human text and a cross-domain transfer test. A short-text split below 30 words and DIPPER-paraphrased machine text provide stress tests, following MGTBench's finding that adversarial perturbation took F1 from 0.99 to 0.04.

Metrics. The primary metric is TPR at 1% FPR, because false accusations of humans are the harm. We also report AUROC and the false-positive rate on non-native writers.

Decisive check 1: privacy risk. A loss-based membership-inference audit against the non-private and DP detectors establishes that the privacy risk to essay authors is real and that DP closes it.

Decisive check 2: detection versus semantics. We compare the fitted scaling slope of detection with that of SST-2 under identical DP training. A clearly flatter slope for detection would support the claim that detection signals lie off the pretrained semantic slice.

Decisive check 3: which ingredient matters. We compare the one-sided cloze detector against the private-threshold zero-shot baseline at $\epsilon \in \{1, 3\}$. That comparison, together with the two ablations, determines whether the contribution is the one-sided mechanism, the cloze formulation, both, or neither.