Which Way Was It Written? Directional Context Fit as a Signature of Machine Rewriting

A detector that asks "does this text look machine-made?" can be defeated by a few human edits or an automated synonym swap. We propose asking something harder to fake: which direction was each sentence written from? When a language model rewrites or inserts a sentence inside a human document, it usually generates from left to right, so the new sentence fits what came before it. The human sentences after it, however, were written to follow the original sentence, not the replacement. The result is a seam that shows up when a model reads the sentence against its right-hand context, and hardly at all when it reads the sentence against its left-hand context.

Our instrument is a single masked language model: an encoder that predicts hidden tokens from the words on both sides. We score each sentence twice, once with only its left context visible and once with only its right context visible, and treat the gap between the two scores as a per-sentence feature. A sequence tagger turns these features into span-level labels for documents with several rewritten regions. The adversarial argument is that an attacker faces a dilemma. A rewrite conditioned on the left leaves a directional seam. A rewrite that erases the seam by conditioning on both sides, such as masked-infilling "humanizers," becomes too predictable under a masked language model, which is a second fingerprint. We test whether this dilemma holds and where it breaks. Expert human editors are the expected point of failure.

Core Claim

The direction in which a text span was conditioned when it was produced is a measurable forensic signal. It is largely independent of the surface statistics that polishing and paraphrase attacks destroy. Using one bidirectional masked model with asymmetric masking makes the signal a controlled comparison: the same model and the same tokens, with only the visible context changed. This lets a sentence-level tagger locate machine-rewritten spans more robustly across generators than fine-tuned encoders that use both sides of the context implicitly.

For sentence $s_i$ with left context $L_i$ and right context $R_i$, the directional fit asymmetry is

$$A_i = \frac{1}{|s_i|}\Big[\mathrm{PLL}(s_i \mid L_i) - \mathrm{PLL}(s_i \mid R_i)\Big],$$

where $\mathrm{PLL}(s_i \mid C)$ is the masked model's pseudo-log-likelihood of $s_i$: each token is masked in turn and scored while only context $C$ is visible and the other side is fully masked. $|s_i|$ is the token count. We also compute a two-sided predictability $B_i = \mathrm{PLL}(s_i \mid L_i, R_i)/|s_i|$ and a downstream-dependence term. The latter measures how much the score of $s_{i+1}$ drops when $s_i$ is masked, which tests whether the following human sentence actually "leans on" $s_i$.

The hypotheses are specific. Left-conditioned machine spans should show positive $A_i$ and weak downstream dependence. Spans rewritten with a masked model on both sides should show near-zero $A_i$ but unusually high $B_i$. Human spans should show neither pattern.

Grounding in Prior Work

Three findings from the evasion literature motivate the problem. Localized machine intervention degrades detectors: DetectRL reports a 13.19% detection drop when 25% of AI text is replaced with human text. Beemo finds that expert edits of 20–40% reduce AUROC by up to 22%, while LLM-based humanization stays easy to flag. Almost AI, Almost Human shows the reverse failure: GLTR's false positive rate rises from 6.83% to 40.87% on human text given only extremely minor GPT-4o polish. These results argue for span-level attribution instead of whole-document verdicts. A Practical Examination argues that the right metric for this is TPR at 1% false positive rate, not AUROC, and we adopt it.

Three papers constrain the design because they already use bidirectional context for localization. AdaLoc freezes RoBERTa and labels chunks of three sentences jointly, raising GoodNews mAP from 50.67 to 78.49. It shows that neighboring context matters, but it never separates left fit from right fit and it tests only older generators. DeBERTa-CRF tags a single human-to-machine boundary with MAE 2.192 on M4GT-bench, but its MAE rises to 18.538 on unseen LLaMA2 text. RoFT boundary detection finds that fine-tuned RoBERTa loses 6% accuracy on ChatGPT continuations, while perplexity features transfer better. Together these suggest that a small number of interpretable, model-agnostic scores may transfer across generators better than end-to-end fine-tuning. Our proposal differs from all three because the bidirectional model is used as a measurement instrument with controlled masking, not only as a feature encoder. It also targets multiple, non-contiguous rewritten spans instead of one cut point.

The method comes from bidirectional generation. SENSESHIFT shows that causal decoders produce in-text rewrites that fit poorly with the sentences that follow, while iterative ModernBERT infilling fits both sides. That result is the empirical basis for our seam hypothesis, and we reuse its infilling procedure to build the strongest adaptive attacker. HMGC uses BERT masked infilling as an attack, reducing a white-box RoBERTa detector's AUC from 99% to 51.06%, and adversarial retraining plateaus at about a 10% failure rate. We treat HMGC as the prototype of the second branch of the dilemma: two-sided infilling should leave excess masked-model predictability. Linear-time bidirectional recurrence (BLUR, DeltaFlow) plays only a minor role. Our tagger operates over sentences, and sentence sequences are short, so we make no efficiency claim.

Research Program

The system has three parts. First, a feature extractor built on ModernBERT computes $A_i$, $B_i$ and downstream dependence for every sentence, using left and right context windows of up to five sentences. For comparison it also computes a causal-LM perplexity feature. Second, a light bidirectional recurrent tagger with a conditional random field on top reads these features, optionally concatenated with frozen encoder embeddings, and outputs human or machine labels per sentence. The conditional random field is a decoder that scores whole label sequences. It permits any number of boundaries and learns how labels transition from one sentence to the next. Third, an attack suite produces rewrites at matched edit budgets under three conditioning regimes. Left-conditioned rewrites come from an instruction-tuned LLM that sees only the preceding text. Full-context LLM rewrites come from a model that sees the whole paragraph. Two-sided rewrites come from SENSESHIFT-style iterative ModernBERT infilling and HMGC synonym attacks. Each attack is also applied in an adaptive version that directly minimizes $|A_i|$, or both $|A_i|$ and $B_i$, using the defender's own scorer.

Because Beemo and APT-Eval provide aligned original and edited text, we can build span labels by diffing the versions. This yields multi-boundary test sets with real human edits and real LLM polish, which the single-boundary benchmarks lack.

Baselines

The naive reference point labels each sentence with a zero-shot causal score. We use per-sentence Binoculars, reported as the most robust zero-shot method on DetectRL with an average AUROC of about 79.6% (DetectRL), and plain causal perplexity. It is run on exactly the same sentence segmentation. It represents what practitioners do today.

The strongest localization competitor is AdaLoc with RoBERTa-Large, which reports 78.49 mAP on GoodNews. We rerun it with the released recipe on the same training splits, tuning trials and GPU budget as our tagger. On GoodNews we first check that our rerun reproduces roughly the reported number. Beating it means higher sentence mAP in-domain and a larger margin under cross-generator and adversarial transfer.

For single-boundary data we compare against DeBERTa-CRF, which reports MAE 2.192 on M4GT-bench dev and 18.538 on LLaMA2 test. The meaningful win is a substantially lower unseen-generator MAE, not the in-domain figure.

For document-level four-way attribution, D2C-Routing reports an average TPR@1%FPR of 0.8603 on MixD2C. We add our aggregated span features to its fusion and ask whether they help, especially in the separation of humanized AI text from fully AI text, which D2C identifies as its bottleneck.

The ablation that isolates the key ingredient keeps the tagger, data and budget fixed. It removes the directional masking and gives the tagger only $B_i$ plus encoder features, which amounts to "bidirectional context without direction." A second ablation replaces $A_i$ with the difference between a causal LM score and a masked-model score. This checks that the gain comes from controlled asymmetry and not from simply combining two models, as a Binoculars-style ratio would.

Why not just…?

Why not just fine-tune a bigger bidirectional encoder tagger, since DeBERTa already sees both sides? This is the most serious threat. In principle an encoder can learn directional fit implicitly, and in-domain it may match us. The evidence that it fails is about transfer: RoBERTa loses 6% on RoFT-chatgpt (RoFT), and DeBERTa-CRF's MAE rises roughly eightfold on LLaMA2. The feature-ablation experiment settles the question. If the full-context encoder tagger matches the asymmetry tagger on held-out generators and on the adaptive attacks, the explicit asymmetry adds nothing, and we will report that.

Why not just use causal perplexity per sentence? A causal model sees a left-conditioned seam only as surprise at the next sentence, so it blames the wrong sentence. RoFT reports that ordinary narrative shifts also cause perplexity spikes that trigger false boundaries. Metric-based detectors score about 0.25 F1 on MixSet three-way classification (LLM-as-a-Coauthor). The naive baseline and the causal-difference ablation measure this directly.

Why not just adversarially train on the attacks? HMGC reports that retraining plateaus at about a 10% failure rate against its attack. We include an adversarially trained AdaLoc as an extra arm. Our claim is that directional features lose less performance on attack families held out from training, which retraining cannot provide by construction.

Why not just ask a frontier LLM where the edits are? RoFT reports that prompting GPT-3.5 and GPT-4 for boundaries does clearly worse than specialized classifiers. We run one prompted arm on the multi-boundary Beemo diffs as a sanity check and do not expect it to compete.

Decisive Experiments

The first check is a mechanistic test run before any training. On DetectRL mixing, AdaLoc's datasets and RoFT-chatgpt, the distribution of $A_i$ for machine sentences should shift positive relative to human sentences. On our controlled attack suite, the shift should shrink from left-conditioned to full-context to two-sided rewrites, while $B_i$ rises for masked-model rewrites. If $A_i$ does not separate left-conditioned insertions from human text, the proposal fails at its foundation.

The second check is transfer. We train on the older generators in GoodNews and RoFT and test on RoFT-chatgpt, M4GT LLaMA2, Beemo's ten generators and APT-Eval's DeepSeek-V3 and GPT-4o polish. Metrics are sentence mAP, span F1, boundary MAE, and TPR@1%FPR at both sentence and document level.

The third check is the dilemma under adaptive attack. For each attacker that minimizes the defender's features, we plot detection against semantic similarity, measured with the same Universal Sentence Encoder threshold of at least 0.85 that HMGC uses. The claim survives if no attacker drives both $A_i$ and $B_i$ to human levels without losing similarity or fluency.

The fourth check targets the case we expect to lose. Beemo's expert edits and APT-Eval's full-paragraph polishing produce no left-to-right seam. We report performance there separately, together with false positive rates on the pure human APT-Eval texts, so that the method's scope is stated honestly rather than averaged away.