Analyzing Code Provenance in LLMs for Programming Education

Subhankar Maity, Faiza Belbachir, Omar Boucelma, Guilherme Medeiros MachadoarXiv 2026 · arXiv

This paper, "Analyzing Code Provenance in LLMs for Programming Education," presents a framework for identifying the origins of source code in educational settings. By extending membership inference techniques, the authors aim to help instructors distinguish between student-authored logic and code "memorized" or generated by Large Language Models (LLMs).

TL;DR

The authors extend the Min-K% membership inference attack to Python code, creating a tool that detects whether code snippets were likely part of an LLM's training data. Their findings reveal a significant trade-off: while free, open-source models are highly susceptible to provenance detection, commercial models (like GPT-5 and Gemini) employ privacy safeguards that make their "fingerprints" harder to trace.

What Problem Does This Paper Address?

The rapid adoption of LLMs has disrupted traditional programming education. When a student can generate a fully functional solution instantly, the final "product" (the code) ceases to be a reliable indicator of the "process" (the learning).

Traditional plagiarism detection tools (like MOSS or JPlag) compare student submissions against each other or known repositories, but they struggle to identify code generated directly by an AI. The authors argue that we need code provenance: the ability to trace the origin and evolution of code segments. This isn't just about catching "cheaters"; it’s about giving instructors visibility into which concepts students are outsourcing to AI (e.g., data loading vs. complex recursion) so they can provide better formative feedback.

Methods

The researchers built their pipeline on an extended version of the Min-K% membership inference attack.

  1. The Core Intuition: LLMs assign higher probabilities (lower perplexity) to sequences they have seen during training. The Min-K% approach focuses on the $k$ percent of tokens with the lowest probabilities, as these "uncertain" tokens are often the most discriminative for identifying training data membership.
  2. Synthetic Non-Member Generation: To train and validate their detector, the authors needed a "negative" dataset of code the models definitely hadn't seen. They introduced a novel method using five transformation strategies to create synthetic variations of existing code:
  3. Semantic: Renaming variables/functions.
  4. Structural: Altering indentation and control flow.
  5. Aggressive: Injecting redundant logic or breaking code while maintaining syntax.
  6. Cross-Language: Converting Python to JavaScript, C++, etc.
  7. Novel Patterns: Generating distant structures like SQL or Docker configs.
  8. The Workflow: As shown in the provided figure, the system tokenizes student code, queries a target LLM to get token probabilities, and applies the Min-K% score against a calculated threshold to flag code as "Member" (AI-influenced) or "Non-member" (Authentic).
Fig. 1. Workflow for detecting LLM-generated code in students’ code using the Min- K% membership inference attack.

Datasets

  • ProCQA: A massive dataset containing ~1.008 million Python records sourced from Stack Overflow. This served as the "member" data (representative of what LLMs are trained on).
  • Synthetic Data: Created by applying the five transformation strategies to the ProCQA samples.
  • Student Assignments: Real-world data from a data science hackathon where students predicted Titanic passenger survival. Crucially, students provided their LLM interaction logs, which served as the "ground truth" for provenance.
Table 1. Distribution of key Python programming concepts and the number of pro- gramming questions related to each concept in the ProCQA dataset.

Key Insights and Ideas

  • Model Size vs. Privacy: Interestingly, model size is not a direct predictor of "leakiness." Some smaller free models (like DistilGPT-2) were easier to detect than massive commercial ones. This suggests that commercial providers are successfully integrating privacy-preserving mechanisms like differential privacy or noise injection.
  • The "Pedagogical Filter": The study found that students primarily used LLMs for "boilerplate" tasks like package imports and feature engineering. By identifying these segments, instructors can focus their grading on the unique problem-solving logic students actually wrote themselves.
  • Attribution Accuracy: In the student hackathon test, the system consistently attributed the highest likelihood of generation to GPT-5 (ChatGPT), which perfectly matched the students' self-reported logs.

Key Results

  • Free Models: DistilGPT-2 was the most detectable, achieving an Area Under the Curve (AUC) of 0.713. GPT-2 showed the highest sensitivity at low false-positive rates (TPR@5%FPR of 18.19%).
  • Commercial Models: Performance was generally lower, indicating better privacy. Gemini-2.5-Flash led this group with an AUC of 0.654, while GPT-5 had the best TPR@5%FPR at 10.14%.
  • Student Data Validation: On real-world student code, the method achieved AUCs between 0.58 and 0.64. While lower than the controlled Stack Overflow tests, it maintained high precision (up to 0.895 for some groups), meaning when it flagged code as AI-generated, it was usually right.

Limitations and Future Work

The authors acknowledge a significant hurdle: conflation. Because LLMs are trained on public data like Stack Overflow, the tool might flag code that a student manually copied from a website as "AI-generated." While both represent a reliance on external sources, the cognitive processes are different.

Future work aims to "disentangle" specific provenance—moving beyond just saying "this looks like training data" to identifying exactly which model or web source the code originated from. Additionally, the authors want to improve detection sensitivity for code that has been heavily edited by students after generation.

Summary updated

Abstract

Large language models (LLMs) play an increasing role in programming education, challenging traditional assessment methods as they can generate fully functional code instantly. This disrupts the reliability of student submissions as indicators of learning and effort. To address this issue, code provenance is essential for tracing the origins of code and distinguishing student-authored, AI-generated, and externally sourced components. We extend the Min-K% probability approach by incorporating synthetic non-member code generation and apply it to Python code for membership inference and provenance tracking. Evaluations cover free LLMs such as GPT-2, DistilGPT-2, and DialoGPT, as well as state-of-the-art commercial models including GPT-5, Gemini 2.5, Mistral, DeepSeek, and LLaMA 3.3. Before evaluating our approach on higher-education student assignments, we validate its feasibility using Stack Overflow data. The results show that free models enable more sensitive code memorization detection than commercial models. By systematically comparing these LLMs, we assess trade-offs between detection accuracy and privacy protection, taking an initial step toward effective assessment of AI-assisted programming education through combined provenance detection and process visibility.
BibTeX
@misc{maity2026analyzing,
  author = {Subhankar Maity and Faiza Belbachir and Omar Boucelma and Guilherme Medeiros Machado},
  title = {{Analyzing Code Provenance in LLMs for Programming Education}},
  year = {2026},
  eprint = {2611.09177},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09177},
}
Figure