Detectors for machine-written text work well on essays and fail on tweets. The usual explanation is sample size: a 30-word post holds too few tokens for a language model's small stylistic biases to add up. This proposal argues that part of the failure comes from something more fixable. Most zero-shot detectors measure how predictable a text is, and on a short text that number is dominated by what the text is about (rare names, topic, register) rather than by who wrote it. Condensed-matter physics faces the same problem when it hunts for a subtle broken symmetry under a large background signal. Its standard answer is to measure a quantity that symmetry forces to be exactly zero unless the hidden order is present. Second harmonic generation is the classic case. It is light at twice the input frequency, and it is forbidden in any crystal with inversion symmetry. Any signal at all is therefore evidence of broken symmetry, with no background to subtract.
We propose the textual equivalent. Score a short text with a left-to-right language model and with a matched right-to-left model trained on the same data. Then use the difference between the two log-probabilities as the detector. For an ideal model of language the two directions must agree exactly, because the chain rule factorizes the same joint probability either way. Content-driven surprisal (a rare word, an odd topic) is priced similarly in both directions and largely cancels. What survives is direction-specific: text produced by a forward model under truncated decoding is selected to sit where forward models' shared errors overrate it, and backward models do not share that selection. The claim is that this odd, background-cancelling channel keeps discriminative power at lengths of 10–35 words, where perplexity, curvature and syntax methods have already collapsed.
Core Claim
Let $x$ be a text of $n$ tokens. Let $\log p_{\rightarrow}(x)$ and $\log p_{\leftarrow}(x)$ be its total log-probability under a forward and a backward model trained on identical data with identical tokenizer and size. We define the asymmetry score
$$ A(x) = \frac{1}{n}\left[\log p_{\rightarrow}(x) - \log p_{\leftarrow}(x)\right] - \mu_h, $$
where $\mu_h$ is the mean of the bracketed term on held-out human text. This offset absorbs any systematic gap between the two models, playing the role that surface and quadrupole contributions play in optics.
The claim has three parts. First, on human text, the per-token spread of $A$ is much smaller than the per-token spread of ordinary log-probability, because most content-driven variance is shared by the two directions. Second, machine text from forward generators has positive mean $A$. Third, because the length needed to reach a fixed error rate scales roughly as (spread / gap)², which follows from averaging $n$ weakly dependent per-token contributions, the asymmetry score should reach a given AUROC at much shorter lengths than single-direction scores.
The physics supplies the design rules, not the mechanism. Two further rules carry over. A null-by-symmetry probe should be paired with an ordinary probe, as second and third harmonic measurements were paired to separate distortions from the symmetric background. And a sign-reversal control should flip the signal.
Where This Sits
The length cliff is well documented and motivates the work. Ghostbuster combines weak-model token probabilities and degrades below 100 tokens (Ghostbuster). TOCSIN measures how much a text's meaning shifts when a few tokens are deleted, and its human and machine distributions overlap below 90 tokens (TOCSIN). PRDetect classifies dependency trees and lacks structure under 100 tokens (PRDetect). Retrieval defenses need at least 50 tokens (Krishna et al.). Shared tasks report the same ceiling: in Russian texts averaging 31 tokens, the best system reached 83.0% accuracy (RuATD 2022), and 35-word texts were the hardest of 70 conditions at PAN 2024 (Voight-Kampff). All of these treat brevity as a fixed sample-size penalty. We test whether a large part of that penalty is removable background.
Four lines of work are closest in method. DetectGPT asks whether a text sits at a sharp peak of model probability under random perturbations (DetectGPT); it needs many perturbations and still measures a single direction. Raidar measures how little an LLM edits a text when asked to rewrite it, which is a transformation-invariance signal that works down to 10 words (Raidar). It does so through expensive black-box rewriting calls, whereas our transformation is a fixed, closed-form reversal scored in two forward passes. Telescope finds a local, vestigial aversion to immediate token repetition that survives on short texts (Telescope); it is a local cue in one direction, and we treat it as a complementary channel. Wu et al. prove that concatenating segments amplifies the human–machine gap and use long texts to supervise short ones (Easy-to-Hard Supervision). That accepts the sample-size view, whereas we try to raise the per-token signal-to-noise ratio itself.
The physics papers play the role of a lens:
- In Sr₂IrO₄, a second-harmonic probe revealed an inversion-breaking order that diffraction missed, because the probe has no centrosymmetric background (Zhao et al.).
- Pairing second and third harmonics separated a staggered distortion from the symmetric lattice (Torchinsky et al.).
- The absence of a second-harmonic signal settled a phase question in CaFeTi₂O₆ that X-ray refinement could not (Li et al.).
- Decomposing a scanning-probe signal into symmetric and antisymmetric parts isolated piezoelectric from flexoelectric response at domain walls (Wei et al.).
The Zhao paper also supplies a caution we take seriously. A symmetry-filtered signal can come from a nuisance source (there, higher-order multipoles; here, mismatch between the two models). The calibration and control experiments below exist to rule that out.
Research Program
We first build the instrument. We train matched forward and backward decoder models at two scales, roughly 400M and 1B parameters, on the same public web corpus, with the same tokenizer, schedule and token budget. The backward model sees each document reversed at the token level. To have an ablation with identical training, we also train a second forward model with a different seed.
We then measure the physics-style quantities directly on human text from each benchmark, stratified by length:
- the per-token variance of single-direction log-probability;
- the variance of the asymmetry score;
- their correlation;
- the mean gap on machine text, broken down by generator and decoding temperature.
These measurements test the cancellation claim before any detector is built. If the two directions' surprisals are not highly correlated on human text, the idea fails at this stage and we report that.
Next we build the detector as two channels:
- an "odd" channel, the asymmetry score;
- an "even" channel, the average of the two directions' log-probabilities, analogous to the third-harmonic background probe;
- optionally, Telescope's local repetition score.
The channels are combined by a two-parameter logistic fit on a small calibration set from a single source domain. The detector is otherwise zero-shot.
Finally we test the mechanism through predictions it makes and alternatives do not:
- The gap should grow as generator temperature falls and with truncated (top-p) sampling, since the gap comes from forward-greedy selection.
- The gap should shrink as the scoring models grow more consistent across directions.
- The sign of the score should flip for text produced by a backward sampler and then reversed.
Baselines
The naive reference is average forward log-probability (perplexity) from the same forward model. Telescope reports that this reaches AUROC 0.893 on its Detect LLM Text set against Telescope's 0.992 (Telescope), though without length stratification. We run it on identical length-truncated inputs.
The strongest zero-shot competitors are Binoculars and Fast-DetectGPT. Both already normalize away content-driven surprisal, Binoculars by contrasting two forward models, so they are the direct test of whether reversal adds anything beyond normalization. MultiSocial reports both at AUC ROC 0.72–0.76 on short social posts (MultiSocial). We run them with off-the-shelf models and also re-instantiated on our own matched pair, so that model quality is held fixed. Telescope is run with the same forward model. Raidar is included at 10–50 words on its own domains, with its LLM-call cost reported alongside.
The strongest overall competitor is supervised fine-tuning. MultiSocial reports AUC ROC 0.97–0.98 for fine-tuned Llama-3-8B and Mistral-7B (MultiSocial). RuATD reports a fine-tuned BERT at 79.7% against the top system's 83.0% (RuATD 2022). Wu et al. report RADAR at TPR@1%FPR of 27.98%, rising to 31.65% with their easy-to-hard supervision, on DetectRL (Easy-to-Hard Supervision). We fine-tune a classifier of comparable size to our scoring models on the same training split we allow ourselves for calibration, and also on the full split, and evaluate both in-distribution and on held-out generators and domains.
The key ablation replaces the backward model with the second forward model trained with a different seed. This forward–forward difference cancels content-driven variance in the same way but has no direction. If it matches the asymmetry score, the arrow of time is not the ingredient and the claim is false.
Beating the zero-shot field means three things on the English short-post subsets:
- AUC clearly above the reported 0.76 ceiling; we set 0.85 as the bar;
- a leftward shift of the length-versus-AUROC curve against Binoculars and Telescope at matched model size;
- a margin over the forward–forward ablation that holds across generators.
We do not expect to beat in-distribution fine-tuning. We count it a success if the detector, or the fine-tuned classifier with the asymmetry score added as a feature, beats plain fine-tuning on held-out generators and paraphrased inputs.
Why not just…?
Why not just fine-tune a large classifier, since MultiSocial already reaches 0.97–0.98? In-distribution, that is a real threat and we expect it to win. The case against relying on it is generalization. RoBERTa-style detectors lose 13.8 F1 points under domain shift relative to Ghostbuster (Ghostbuster) and fall to about 50% under 30% synonym substitution (PRDetect). Our held-out-generator and perturbation splits test whether the asymmetry score is more stable. Adding it as a feature to the fine-tuned model tests whether it carries information the classifier does not learn on its own.
Why not just use Binoculars, which already cancels content with two models? This is the sharpest objection. Binoculars contrasts two forward models that share the generator's directional bias, so its normalization may cancel part of the machine signal along with the background. Whether reversal adds anything is exactly what the forward–forward seed ablation and the matched-model Binoculars comparison measure. If they tie, we drop the directional claim.
Why not just use Telescope's local cue? Telescope already survives short lengths, and its failure modes (repetitive genres, prompting the generator to repeat) differ from ours. We include it as a third channel and report whether the asymmetry score adds to it, especially on poetry-like and templated posts where Telescope is known to fail.
Why not just concatenate short texts? Wu et al.'s amplification needs several segments from the same source. A moderator scoring one post does not have them. Our method targets the single-text case, and we report the concatenation regime separately for completeness.
Decisive Checks
We evaluate on:
- the English subset of MultiSocial, plus a small multilingual model pair if budget allows;
- the PAN 2024 35-word variant, for which no reported result specific to the 35-word condition was found on Lacuna; the winning ensemble's 0.961 ROC-AUC covers all 70 variants;
- DetectRL, truncated to 20/35/50/100 tokens;
- Raidar's short-input domains;
- RuATD, with a Russian model pair, as a stretch goal.
Metrics are AUROC and TPR at 1% FPR, plotted against length.
Three outcomes would settle the question. First, on human text the asymmetry score must show a markedly lower per-token variance than log-probability; otherwise the background-cancellation premise is wrong. Second, the score must beat the forward–forward ablation; otherwise direction is irrelevant. Third, text sampled from the backward model and reversed must produce a significantly negative score. This sign-flip control is the analog of reversing a domain: it separates a genuine directional selection effect from a mismatch artifact between the two models. Any surviving artifact would show up as a nonzero shift on human text that calibration cannot remove across domains.
We also expect the gap to shrink at temperature 1 with no truncation. That prediction tells deployers when the detector can be evaded, and we report it as such.
References
- Ghostbuster: Detecting Text Ghostwritten by Large Language Models
- Zero-Shot Detection of LLM-Generated Text using Token Cohesiveness
- PRDetect: Perturbation-Robust LLM-generated Text Detection Based on Syntax Tree
- Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
- Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian
- Overview of the "Voight-Kampff" Generative AI Authorship Verification Task at PAN and ELOQUENT 2024
- DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
- Raidar: geneRative AI Detection viA Rewriting
- Telescope: Improving Zero Shot Detection of LLM Generated Content By Measuring Token Repetition Probability
- Advancing Machine-Generated Text Detection from an Easy to Hard Supervision Perspective
- Evidence of an odd-parity hidden order in a spin–orbit coupled correlated iridate
- Structural Distortion-Induced Magnetoelastic Locking in<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:msub><mml:mrow><mml:mi>Sr</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msub><mml:msub><mml:mrow><mml:mi>IrO</mml:mi></mml:mrow><mml:mrow><mml:mn>4</mml:mn></mml:mrow></mml:msub></mml:mrow></mml:math>Revealed through Nonlinear Optical Harmonic Generation
- Spin freezing into a disordered state in <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mi>CaFeT</mml:mi><mml:msub><mml:mi mathvariant="normal">i</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:msub><mml:mi mathvariant="normal">O</mml:mi><mml:mn>6</mml:mn></mml:msub></mml:mrow></mml:math> synthesized under high pressure
- Co-Dominant Piezoelectric and Flexoelectric Effects in Twisted Double Bilayer Graphene
- MultiSocial: Multilingual Benchmark of Machine-Generated Text Detection of Social-Media Texts