This paper, "Analyzing Code Provenance in LLMs for Programming Education," presents a framework for identifying the origins of source code in educational settings. By extending membership inference techniques, the authors aim to help instructors distinguish between student-authored logic and code "memorized" or generated by Large Language Models (LLMs).
TL;DR
The authors extend the Min-K% membership inference attack to Python code, creating a tool that detects whether code snippets were likely part of an LLM's training data. Their findings reveal a significant trade-off: while free, open-source models are highly susceptible to provenance detection, commercial models (like GPT-5 and Gemini) employ privacy safeguards that make their "fingerprints" harder to trace.
What Problem Does This Paper Address?
The rapid adoption of LLMs has disrupted traditional programming education. When a student can generate a fully functional solution instantly, the final "product" (the code) ceases to be a reliable indicator of the "process" (the learning).
Traditional plagiarism detection tools (like MOSS or JPlag) compare student submissions against each other or known repositories, but they struggle to identify code generated directly by an AI. The authors argue that we need code provenance: the ability to trace the origin and evolution of code segments. This isn't just about catching "cheaters"; it’s about giving instructors visibility into which concepts students are outsourcing to AI (e.g., data loading vs. complex recursion) so they can provide better formative feedback.
Methods
The researchers built their pipeline on an extended version of the Min-K% membership inference attack.
- The Core Intuition: LLMs assign higher probabilities (lower perplexity) to sequences they have seen during training. The Min-K% approach focuses on the $k$ percent of tokens with the lowest probabilities, as these "uncertain" tokens are often the most discriminative for identifying training data membership.
- Synthetic Non-Member Generation: To train and validate their detector, the authors needed a "negative" dataset of code the models definitely hadn't seen. They introduced a novel method using five transformation strategies to create synthetic variations of existing code:
- Semantic: Renaming variables/functions.
- Structural: Altering indentation and control flow.
- Aggressive: Injecting redundant logic or breaking code while maintaining syntax.
- Cross-Language: Converting Python to JavaScript, C++, etc.
- Novel Patterns: Generating distant structures like SQL or Docker configs.
- The Workflow: As shown in the provided figure, the system tokenizes student code, queries a target LLM to get token probabilities, and applies the Min-K% score against a calculated threshold to flag code as "Member" (AI-influenced) or "Non-member" (Authentic).
Datasets
- ProCQA: A massive dataset containing ~1.008 million Python records sourced from Stack Overflow. This served as the "member" data (representative of what LLMs are trained on).
- Synthetic Data: Created by applying the five transformation strategies to the ProCQA samples.
- Student Assignments: Real-world data from a data science hackathon where students predicted Titanic passenger survival. Crucially, students provided their LLM interaction logs, which served as the "ground truth" for provenance.
Key Insights and Ideas
- Model Size vs. Privacy: Interestingly, model size is not a direct predictor of "leakiness." Some smaller free models (like DistilGPT-2) were easier to detect than massive commercial ones. This suggests that commercial providers are successfully integrating privacy-preserving mechanisms like differential privacy or noise injection.
- The "Pedagogical Filter": The study found that students primarily used LLMs for "boilerplate" tasks like package imports and feature engineering. By identifying these segments, instructors can focus their grading on the unique problem-solving logic students actually wrote themselves.
- Attribution Accuracy: In the student hackathon test, the system consistently attributed the highest likelihood of generation to GPT-5 (ChatGPT), which perfectly matched the students' self-reported logs.
Key Results
- Free Models: DistilGPT-2 was the most detectable, achieving an Area Under the Curve (AUC) of 0.713. GPT-2 showed the highest sensitivity at low false-positive rates (TPR@5%FPR of 18.19%).
- Commercial Models: Performance was generally lower, indicating better privacy. Gemini-2.5-Flash led this group with an AUC of 0.654, while GPT-5 had the best TPR@5%FPR at 10.14%.
- Student Data Validation: On real-world student code, the method achieved AUCs between 0.58 and 0.64. While lower than the controlled Stack Overflow tests, it maintained high precision (up to 0.895 for some groups), meaning when it flagged code as AI-generated, it was usually right.
Limitations and Future Work
The authors acknowledge a significant hurdle: conflation. Because LLMs are trained on public data like Stack Overflow, the tool might flag code that a student manually copied from a website as "AI-generated." While both represent a reliance on external sources, the cognitive processes are different.
Future work aims to "disentangle" specific provenance—moving beyond just saying "this looks like training data" to identifying exactly which model or web source the code originated from. Additionally, the authors want to improve detection sensitivity for code that has been heavily edited by students after generation.
Summary updated