Abstract
The widespread use of large language models (LLMs) is increasing the demand for methods that detect machine-generated text to prevent misuse. The goal of our study is to stress test the detectors' robustness to malicious attacks under realistic scenarios. We comprehensively study the robustness of popular machine-generated text detectors under attacks from diverse categories: editing, paraphrasing, prompting, and co-generating. Our attacks assume limited access to the generator LLMs, and we compare the performance of detectors on different attacks under different budget levels. Our experiments reveal that almost none of the existing detectors remain robust under all the attacks, and all detectors exhibit different loopholes. Averaging all detectors, the performance drops by 35% across all attacks. Further, we investigate the reasons behind these defects and propose initial out-of-the-box patches to improve robustness.
Introduction
As Large Language Models (LLMs) become ubiquitous, distinguishing between human-written and machine-generated text (MGT) is critical for academic integrity and preventing disinformation. However, most detectors are evaluated in "clean" environments. In reality, malicious actors will actively attempt to bypass these systems. This study systematically evaluates detector robustness across a wide range of attack categories to identify critical vulnerabilities.
Methodology: A Comprehensive Stress Test
The research framework evaluates detectors against 12 realistic attack strategies categorized into four stages of the text generation pipeline:
- Editing (Post-generation): Character-level changes like typos, homoglyphs (visually similar characters), or zero-width spaces.
- Paraphrasing (Post-generation): Rewriting text using other LLMs or dictionaries.
- Prompting (Pre-generation): Instructing the LLM to mimic human styles or substitute characters during generation.
- Co-generating (On-generation): A novel attack where the generator inserts noise (like emojis) during sampling, which is later removed to hide the machine signature.
To ensure attacks remain practical, the authors use a budget metric (e.g., Edit Distance or BERTScore) to ensure the resulting text remains readable and preserves its original meaning.
Experimental Setup
The study utilized news-style datasets and a variety of LLMs, including open-source models (GPT-2 XL, GPT-J, LlaMA-2) and closed-source models (Text-Davinci-003, GPT-4). Evaluation was performed on a balanced set of 2,000 samples (1,000 human, 1,000 machine).
Key Results and the Robustness Leaderboard
The experiments revealed a significant "robustness gap" across different detector types.
The Fragility of Statistical Signatures
Metric-based detectors, such as GLTR and DetectGPT, proved highly vulnerable. For instance, as few as 2 to 6 typos in a 120-token text can degrade their performance to near-random levels. These detectors rely on local token probabilities that are easily disrupted by minor character-level edits.
Comparative Robustness
- Watermarking: Remained the most robust method (~99% relative AUC) but requires "white-box" access to the generator, limiting its use for third-party models like ChatGPT.
- Fine-tuned Models: Classifiers like DeBERTa showed better resilience than metric-based methods because they learn broader semantic patterns rather than local statistics.
- Leaderboard Rankings: Watermarking topped the leaderboard, followed by SimpleAI and OpenAI's large detectors. On average, detector performance dropped by 35% under attack.
Improving Robustness
The authors proposed a simple "out-of-the-box" patch for DetectGPT. By identifying and filtering out extremely low-probability tokens (often the result of typos or homoglyphs), they were able to recover detection performance by an average of 0.2285 AUC.
Conclusion
Current MGT detectors are surprisingly fragile in adversarial settings. While watermarking offers a strong defense, its deployment limitations necessitate the development of more robust, "black-box" detection methods. Future work should focus on cross-lingual robustness and more sophisticated human-in-the-loop attacks.
Summary updated