Zero-shot detectors of machine-generated text rest on one observation: a language model's own outputs sit near peaks of its probability landscape, so small rewrites of a machine passage lower its likelihood far more than the same rewrites of a human passage. This works well on whole documents. It fails on the documents people now actually produce, where a human writes some sentences and an AI assistant writes others. On mixed texts, document-level scores like DetectGPT fall to chance, and even supervised segmenters struggle when the machine-written stretches are only a few sentences long. Scoring each sentence separately does not rescue the curvature signal, because a single sentence is too short for its likelihood drop to rise above noise.
This note proposes looking at a different part of the curvature: the cross terms between sentences, not the diagonal. When a model writes sentence $u$ after sentence $v$, sentence $u$ sits at a peak of the probability conditioned on $v$. Perturbing $v$ should then knock $u$ off its peak. A human sentence that follows has no such dependence. Measuring these sentence-to-sentence effects turns a document into a weighted graph. In that graph, a machine-written span should appear as a group of sentences that are strongly linked to each other and weakly linked to the outside. In graph terms, it has high internal density and a low cut ratio, meaning few strong edges cross its boundary. Finding machine spans then becomes a subgraph-detection problem. The graph-learning literature has shown that this kind of problem needs explicit node marking: a standard graph network cannot compute boundary quantities like cut ratio unless the candidate nodes are labeled.
Core Claim
Our central claim is that the off-diagonal part of a language model's log-probability curvature carries localization evidence that the per-sentence diagonal does not. That evidence also scales better with span length. A machine span of $l$ sentences gives about $l$ noisy diagonal measurements, but it gives about $l^2/2$ pairwise ones. If the pairwise effects are even weakly informative, pooling them should make short spans, where current methods fail, separable. The claim has a theoretical and a practical half. The theoretical half is that the right detector for such spans must be sensitive to subgraph boundaries. The practical half is a two-stage method. A zero-shot spectral partition of the curvature graph comes first, and a small marked-subgraph network trained on top of it comes second.
The edge weight is a cross-sentence perturbation discrepancy. Let $L_u(x) = \log p_\theta(x_u \mid x_{<u})$ be the scoring model's log-probability of sentence $u$ given everything before it. Let $q(\cdot \mid x_v)$ be a meaning-preserving rewrite of sentence $v$ alone, such as span masking and T5 refilling as in DetectGPT. For $v < u$ we define
$$w(u,v) = \frac{L_u(x) - \mathbb{E}_{\tilde{x}_v \sim q}\,[L_u(x_{v \to \tilde{x}_v})]}{\sigma_{u,v}},$$
where $x_{v \to \tilde{x}_v}$ is the document with sentence $v$ replaced. The term $\sigma_{u,v}$ is the standard deviation of the perturbed scores, which normalizes the edge as in Fast-DetectGPT. When $u = v$ the formula reduces to a per-sentence DetectGPT score. Off the diagonal it is a finite-difference cross term.
Relation to Prior Work
The method borrows directly from the curvature-detection papers. DetectGPT defines the perturbation discrepancy and reports AUROC 0.95 on GPT-NeoX news. Fast-DetectGPT replaces sampling with an analytic conditional curvature and normalizes by its variance. Both collapse the signal into one number per document. We keep their perturbation operator and their normalization, but we read out the full sentence-by-sentence matrix instead of its average.
Three papers motivate the localization target. LLM-as-a-Coauthor shows that metric-based detectors fall to roughly 0.25 F1 on three-way human/AI/mixed classification. Towards Detecting AI-Generated Text within Human-AI Collaborative Hybrid Texts shows supervised segmenters degrading on short hand-offs. Machine-Generated Text Localization (AdaLoc) localizes spans with supervised sliding windows over a frozen RoBERTa. None of the three uses likelihood interactions between sentences.
Two graph papers supply the structural machinery. GLASS proves that plain message passing cannot compute cut ratio or internal density without a zero-one label marking the candidate subgraph. It reports a 48.6% gain on cut ratio from that labeling, and our detector inherits the labeling trick for exactly this reason. Nested Graph Neural Networks supplies rooted subgraph encoding with distance features.
On The Expressive Power of GNN Derivatives provides the conceptual bridge. It shows that how strongly perturbing node $v$ affects node $u$ acts as continuous node marking and lets a network detect triangles. Our edges are a discrete analogue of that idea. The difference is that we differentiate the external language model's log-probability rather than the graph network's internal representation.
Two other papers are the closest neighbors, and we should be clear about what separates us from them. Text Graph Neural Networks for Detecting AI-Generated Content builds word co-occurrence graphs, runs a standard attention-based graph network on them, and does whole-document classification. Its edges describe the text's surface statistics, not the generator's likelihood. Text Has Curvature measures how prefix and suffix context reconcile inside a masked language model and applies that to prompt compression. Its curvature is a single-slot quantity and is not used for detection.
Research Program
The first stage builds the curvature graph. For a document of $n$ sentences, we perturb each sentence $k$ times. Each perturbed document needs one forward pass of the scoring model, and that pass returns $L_u$ for every later sentence $u$ at once. The cost is therefore $nk$ forward passes. A 20-sentence essay with $k=10$ needs 200 passes, which is comparable to DetectGPT's roughly 100 per document. We will also test a cheaper variant that substitutes Fast-DetectGPT's analytic sampling inside sentence $v$ for T5 rewriting. Because the scoring model is causal, the raw matrix is lower-triangular. We symmetrize it and keep the diagonal as node features.
The second stage is a zero-shot detector. We look for the subgraph that maximizes internal edge weight relative to boundary weight, which is a normalized-cut problem. Because spans in text are contiguous, we restrict candidates to contiguous intervals. That restriction lets us solve the problem exactly by dynamic programming in $O(n^2)$ instead of with an approximate spectral method. The output is a span score that needs no training.
The third stage is a lightly supervised marked-subgraph network. For each candidate interval, we mark its sentences with a binary label in the GLASS style and add each sentence's distance to the interval boundary in the NGNN style. We then run a small message-passing network over the curvature graph and score the interval. This stage exists for one reason: the normalized-cut objective assumes that machine spans look like clean dense blocks. Real co-authoring breaks that assumption, for instance when a human lightly edits an accepted suggestion. A learned boundary-aware scorer can absorb such patterns, and the theory in GLASS says an unmarked network could not.
The analysis track tests the premise directly. On controlled insertions, we measure the mean and spread of $w(u,v)$ separately for machine→machine, human→machine, machine→human, and human→human pairs, as a function of how far apart the two sentences are. A worry we take seriously is that ordinary topical coherence also makes human sentences depend on earlier ones. The method only works if machine→machine edges stand out beyond that coherence baseline, and the analysis track tests exactly that.
Baselines
The naive reference point is per-sentence Fast-DetectGPT followed by thresholding. It uses the same scoring model and gives each sentence its own label. Prior evidence suggests it will be near chance on short spans. Its role is to establish how much signal the diagonal carries on its own.
The key ablation is the diagonal-only structured detector. It takes the same per-sentence curvature scores and segments them with an optimal change-point / hidden-Markov segmentation. That model is tuned on the same validation split and gets the same perturbation budget. Because it uses identical model calls and differs only in discarding the off-diagonal entries, it isolates the proposal's key ingredient. To beat it, the zero-shot cut detector must show a statistically significant gain in span-level F1, measured over at least three seeds of perturbation sampling, specifically in the short-span regime of $l \le 3$ sentences.
The strongest supervised competitors are the published localizers. The first is AdaLoc (Machine-Generated Text Localization), which reports 78.49 mAP on GoodNews against 50.67 for base RoBERTa-Large. The second is the DeBERTa-v3 segmenter from Towards Detecting AI-Generated Text within Human-AI Collaborative Hybrid Texts. It reports a mean Kappa of 0.3447 on CoAuthor overall, falling to 0.2197 on short-segment documents.
On MIXSET, the strongest reported detector is Radar (7B), at 0.77–0.88 F1 for three-way classification (LLM-as-a-Coauthor). That result is for document classification, not span localization; for span-level localization on MIXSET, no reported result was found on Lacuna. We will rerun AdaLoc and the DeBERTa segmenter on our splits with their published training budgets. Our supervised stage gets an equal or smaller labeled set and a matched hyperparameter search. The headline claim is the following: the zero-shot detector is competitive with AdaLoc out of domain, and the supervised stage exceeds 0.3447 Kappa on CoAuthor with the largest margin in the $l \le 3$ group.
The structural ablation is a GAT or GIN run over the same curvature graph without the subgraph label. It isolates whether boundary marking matters once the edges are fixed.
Why not just…?
Why not just score each sentence with Fast-DetectGPT and smooth with a change-point model? This is the most serious threat. If the per-sentence curvature is already informative enough, structured smoothing will match us at roughly $k$-times lower cost. The evidence against it is indirect. Metric methods collapse on MIXSET, and MGTBench reports that curvature degrades toward chance on short passages, which suggests the diagonal entries are too noisy to segment reliably. The diagonal-only ablation above settles the question directly. If it matches the cut detector, we will report that the pairwise terms add nothing and that the result is negative.
Why not just fine-tune a sequence segmenter such as DeBERTa or AdaLoc? Supervised detectors break under domain and generator shift. DetectGPT reports them near random on German and medical text. HC3 Plus reports a QA-trained RoBERTa dropping to about 58% on summarization and translation. Even in domain, the CoAuthor segmenters fall to 0.2197 Kappa on short hand-offs. Our cross-domain experiment trains supervised baselines on one source and tests on others, and that experiment decides whether the zero-shot signal transfers where they do not.
Why not just run a standard graph network on the curvature graph, or use spectral clustering alone? GLASS proves that unmarked message passing cannot compute cut ratio, so a plain graph network should underperform marked scoring on boundary decisions. The plain-network ablation tests that prediction in this setting. Spectral or normalized-cut partitioning on its own is in fact our zero-shot stage. If it equals the supervised marked network, the honest conclusion is that the graph construction, not the expressive architecture, is the contribution, and we will state it that way.
Proposed Experiments
The first experiment is a controlled insertion study. We take human documents from XSum and WritingPrompts. We replace contiguous spans of $l = 1$ to $6$ sentences with continuations generated from the human prefix by a known open model, and we score with the same model (white-box) and with a different-family surrogate. The decisive check is a curve of span-F1 against $l$ for the cut detector and the diagonal-only ablation. We predict the gap is largest at small $l$, where pairwise evidence matters most.
The second experiment uses realistic co-authoring data. We evaluate on CoAuthor using its Kappa metric with the published short/medium/long segment groups, and on MIXSET by mapping span outputs to the three-way label. Both are surrogate-scored, because the original generators' log-probabilities are unavailable. We report the white-box-to-surrogate drop, given Fast-DetectGPT's observation that white-box scoring is about 65% more accurate.
The third experiment is cross-domain transfer. Supervised baselines and our supervised stage are trained on GoodNews and tested on the Essay and WP portions of the localization suite. The comparison is against the zero-shot detector, using mAP.
The fourth experiment measures robustness and fairness. We apply a paraphrasing pass restricted to machine spans, the attack that drops F1 from 0.99 to 0.04 in MGTBench. We also report false-positive rates on non-native human writing, the population Fast-DetectGPT flags as at risk.
The premise analysis, which compares machine→machine against human→human edge distributions, runs first. If machine→machine edges do not exceed the coherence baseline at matched distance, the program stops there and reports that negative finding.
References
- DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
- Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature
- LLM-as-a-Coauthor: Can Mixed Human-Written and Machine-Generated Text Be Detected?
- Towards Detecting AI-Generated Text within Human-AI Collaborative Hybrid Texts
- Machine-Generated Text Localization
- GLASS: GNN with Labeling Tricks for Subgraph Representation Learning
- Nested Graph Neural Networks
- On The Expressive Power of GNN Derivatives
- Text Graph Neural Networks for Detecting AI-Generated Content
- Text Has Curvature
- MGTBench: Benchmarking Machine-Generated Text Detection
- HC3 Plus: A Semantic-Invariant Human ChatGPT Comparison Corpus