Spotting AI's Touch: Identifying LLM-Paraphrased Spans in Text

Yafu Li, Zhilin Wang, Leyang Cui, Wei Bi, Shuming Shi, Yue Zhang

Introduction

As Large Language Models (LLMs) become ubiquitous, the line between human and machine writing is blurring. We are no longer just dealing with "pure" AI-generated essays; instead, users frequently use AI to "polish," "refine," or "rewrite" specific sections of their own work. Traditional detectors, which act as document-level classifiers, often fail when a document is a hybrid of human and machine touch.

To address this "needle in a haystack" problem for academic integrity and copyright enforcement, researchers have moved beyond binary "AI vs. Human" detection to a more surgical approach: identifying specific spans within a document that have been paraphrased by an LLM. This new task is called Paraphrased Text-span Detection (PTD).

Figure 1: Comparison between traditional machine-generated text detection and the proposed paraphrased text-span detection.

The PTD Framework and Methodology

The authors propose the PTD framework, which treats detection as a sequence labeling or regression task rather than a simple classification. Given a text, the model assigns a score to every sentence to determine if it was paraphrased and to what extent.

Model Architecture

The system uses Longformer as a backbone, allowing it to process long documents while maintaining a global view of the context. This contextual awareness is essential; the study found that models trained only on isolated sentences performed significantly worse. Detecting AI is often less about the sentence itself and more about how that sentence relates to its neighbors.

Regression Metrics

To train the models, the authors developed a "difference scoring" method using three dimensions:

  • Lexical: Changes in word choice (measured via BLEU).
  • Grammatical: Changes in parts-of-speech patterns.
  • Syntactic: Changes in sentence structure (measured via Tree Edit Distance).

The PASTED Dataset

The authors introduced a new benchmark called PASTED (Paraphrased Text span Detection dataset), consisting of over 83,000 instances. It includes original texts from human and machine sources across various domains such as news, Wikipedia, and scientific writing. The dataset features two paraphrasing styles:

  1. Context-agnostic: Rewriting spans in isolation using ChatGPT.
  2. Context-aware: Using the Dipper model, which considers surrounding sentences to make the paraphrase more seamless.

Key Insights: The "Seam" is the Giveaway

One of the most striking findings is that AI-paraphrased text often fails to integrate perfectly with the original human text. The researchers observed perplexity impulses (spikes in unpredictability) at the boundaries where the AI span begins and ends. This suggests that the transition between human and AI writing styles—the "seam"—is a primary signal for detection. Furthermore, the more sentences an AI paraphrases in a row, the easier it is to catch; single-sentence "polishing" remains the most significant challenge.

Results and Robustness

The PTD models demonstrate exceptional performance and robustness. On in-distribution data, the models achieved an AUROC exceeding 0.97. Even when tested against sophisticated, out-of-distribution GPT-4 prompts designed to evade detection, the AUROC remained strong at 0.94.

Figure 6: AUROC performance of the PTD model across various domains and different source LLMs.

As shown in Figure 6, the model's performance is consistent across diverse domains (e.g., XSum, Yelp, Sci_Gen) and various model sets (e.g., OpenAI, LLaMa, EleutherAI). Beyond accuracy, the models are remarkably resistant to minor human edits, such as simple word replacements or sentence reordering, correctly identifying them as "non-paraphrased."

Conclusion

By shifting the focus to fine-grained, sentence-level detection, the PTD framework provides a powerful tool for identifying AI intervention in hybrid texts. Using the PTD model as a "pre-defense" for traditional AI detectors was shown to improve the recall of machine-generated text from 37.05% to 78.50% in scenarios involving paraphrasing attacks. Future work will likely focus on even more sophisticated contextual analysis as LLMs become better at mimicking specific human writing styles.

Summary updated

Abstract

AI-generated text detection has attracted increasing attention as powerful language models approach human-level generation. Limited work is devoted to detecting (partially) AI-paraphrased texts. However, AI paraphrasing is commonly employed in various application scenarios for text refinement and diversity. To this end, we propose a novel detection framework, paraphrased text span detection (PTD), aiming to identify paraphrased text spans within a text. Different from text-level detection, PTD takes in the full text and assigns each of the sentences with a score indicating the paraphrasing degree. We construct a dedicated dataset, PASTED, for paraphrased text span detection. Both in-distribution and out-of-distribution results demonstrate the effectiveness of PTD models in identifying AI-paraphrased text spans. Statistical and model analysis explains the crucial role of the surrounding context of the paraphrased text spans. Extensive experiments show that PTD models can generalize to versatile paraphrasing prompts and multiple paraphrased text spans. We release our resources at https://github.com/Linzwcs/PASTED.
BibTeX
@inproceedings{li2024spotting,
  author = {Yafu Li and Zhilin Wang and Leyang Cui and Wei Bi and Shuming Shi and Yue Zhang},
  title = {{Spotting AI's Touch: Identifying LLM-Paraphrased Spans in Text}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.findings-acl.423},
}
Figure