TL;DR
This paper tackles the challenge of detecting LLM-assisted cheating in high-stakes English proficiency tests, specifically when test-takers copy-type LLM-generated text with errors. They introduce a framework that combines contrastive learning and self-training to improve detection accuracy on these "copy-typed" samples, outperforming existing methods.
What Problem Does This Paper Address?
Large Language Models (LLMs) like GPT-4 are becoming increasingly sophisticated, raising concerns about their misuse as cheating tools in academic and professional assessments. This is particularly problematic for open-ended writing tasks in language proficiency tests, where LLMs can generate high-quality responses. While methods exist to detect AI-generated text, most are trained on "clean" LLM outputs. However, in real-world cheating scenarios, test-takers often copy-type LLM-generated text, introducing typos, misspellings, and other errors. This "copy-typing" process creates a significant gap between researcher-generated LLM text and actual cheating attempts, making existing detectors less effective. This paper aims to bridge this gap by developing a detector that can accurately identify LLM-generated text even after it has been imperfectly modified by a human.
Methods
The authors propose a framework that enhances existing transformer-based text classifiers. The core innovations are:
- Contrastive Learning (SimCLR): They use a contrastive learning approach, inspired by SimCLR, to train the model. This involves creating pairs of "similar" samples (e.g., an original LLM-generated text and its copy-typed version) and "dissimilar" samples. The goal is to make the model's internal representations of these related texts closer, thus improving its ability to recognize variations introduced by copy-typing.
- Approximated Copy-Typing Process: To generate realistic "copy-typed" LLM samples for training, they first use GPT-4 to generate responses and then employ tools like TextAttack to introduce common errors (typos, misspellings, omissions) and simulate the constraints of a timed test (e.g., cutting off text).
- Self-Training: To further leverage real-world data, they employ a self-training strategy. After an initial model is trained, it's used to predict labels on a large set of unlabeled real-world responses. High-confidence predictions are then used as pseudo-labels to retrain and improve the model. This helps the model adapt to the nuances of actual cheating instances.
Datasets
The primary dataset used is from the Duolingo English Test (DET), a high-stakes online English proficiency test.
- Training/Validation/Test Sets: These were constructed by pairing human-written responses (from certified tests before ChatGPT's release to ensure minimal LLM influence) with approximated copy-typed LLM-generated responses.
- Unlabeled Development Set: A large set of 150,000 responses from test sessions in the latter half of 2023 was used for self-training.
- Real-world Test Data: For evaluation, they used responses from test sessions where cheating was detected by human proctors. This dataset is unlabeled, as the specific method of cheating (LLM-assisted or otherwise) is not always known.
Key Insights and Ideas
- The "Copy-Typing" Gap is Crucial: The paper highlights that the imperfections introduced by human copy-typing are a significant challenge for LLM detection, and models trained only on "clean" LLM outputs struggle with these real-world cheating scenarios.
- Contrastive Learning for Robustness: Applying contrastive learning effectively helps the model become more robust to the variations introduced by copy-typing, making it better at identifying LLM-generated text even with errors.
- Self-Training for Real-World Adaptation: Self-training is a valuable technique for incorporating unlabeled, real-world cheating instances, further improving the model's performance on actual malicious use cases.
- In-Domain Fine-Tuning Matters: The proposed fine-tuned RoBERTa models significantly outperform general-purpose detectors, especially at very low false positive rates (FPR), which are critical for high-stakes tests.
Key Results
The proposed ROBERTactr+st model, which combines contrastive learning and self-training, achieved significant improvements:
- On real-world "copy-typed" LLM-generated samples (from tests with violations), the detection rate (PPR) was improved by 1.7x over the initial fine-tuned RoBERTa-base model at a low false positive rate (FPR) of 0.1%.
- The fine-tuned RoBERTa models consistently outperformed general-purpose detectors like OpenAI Detector and GPTZero on both unmodified and copy-typed LLM-generated samples, particularly at low FPRs.
- Figure 2 and 4 show a consistent upward trend in PPR over time for all models, suggesting an increasing prevalence of LLM-assisted cheating.
Limitations and Future Work
- Assumption of Constant FPR: The analysis assumes that the false positive rate (FPR) remains constant over time. However, the authors acknowledge that linguistic drift, especially with the increasing influence of LLMs, could potentially alter this.
- Inferring Cheating vs. Detection: The paper emphasizes that detecting LLM-generated text does not automatically imply cheating. Test policies need to consider this distinction, and human proctors should use these detectors as a signal rather than definitive proof.
- Future Work: The authors plan to further investigate the characteristics of LLM-assisted cheating, including modeling the copy-typing process more precisely. They also aim to extend the framework to other domains, such as spoken language assessments, where complex post-generation modifications might occur.
Summary updated