Abstract
All methodologies for detecting plagiarism to date have focused on the final digital "outcome", such as a document or source code. Our novel approach takes the creation process into account using logged events collected by special software or by the macro recorders found in most office applications. We look at an author's interaction logs with the software used to create the work. Detection relies on comparing the histograms of multiple logs' command use. A work is classified as plagiarism if its log deviates too much from logs of "honestly created" works or if its log is too similar to another log. The technique supports the detection of plagiarism for digital outcomes that stem from unique tasks, such as theses and equal tasks such as assignments for which the same problem sets are solved by multiple students. Focusing on the latter case, we evaluate this approach using logs collected by an interactive development environment (IDE) from more than sixty students who completed three programming assignments.
Introduction
Traditional plagiarism detection tools, such as Turnitin or MOSS, are fundamentally outcome-based. They analyze the final submission—a paper or a source code file—and search for similarities against a database of existing works. However, these methods are increasingly vulnerable to simple obfuscation techniques like "rogeting" (replacing words with synonyms), reordering code blocks, or renaming variables.
This research proposes a paradigm shift: analyzing the creation process rather than just the final product. By capturing the "fingerprint" of how a work was constructed—through keystrokes, mouse clicks, and command usage—we can identify patterns that are characteristic of honest effort versus those that indicate plagiarism.
The Creation Process Framework
The core of this approach lies in capturing Human-Computer Interaction (HCI) logs during the development of a digital artifact. Whether using a text editor or an Integrated Development Environment (IDE), the software records a sequence of events that represent the author's workflow.
As shown in the framework above, the system monitors the interaction between the creator and the software. An honest student typically exhibits a "messy" process involving frequent edits, undos, navigations, and debug cycles. In contrast, a plagiarist who copies and pastes a solution or performs superficial refactoring produces a log that is either an exact statistical match to another student's or a significant outlier compared to the general population.
Methodology and Data Analysis
The detection strategy utilizes two primary pillars: histogram-based analysis and outlier detection. Interaction logs are treated as frequency distributions of command types, such as InsertString, Delete, RunCommand, and FileOpen.
To identify students who have shared or modified logs, the system employs Pearson product-moment correlation on randomly chosen subsets of commands. This ensures robustness against cheaters who might attempt to "noise up" a stolen log by adding random actions. To catch those who copy code from external sources and manually re-type it, the system calculates an outlier score. By applying a Box-Cox transformation to normalize command frequencies, the system can flag logs that lack essential "honest" behaviors—like frequent navigation or specific file-handling patterns—as statistical anomalies.
Evaluation and Results
The methodology was evaluated using three distinct datasets: a real-world set of logs from over 60 students, a synthetic dataset for simulating various cheating intensities, and a set of manually created "fake" logs representing different refactoring strategies.
Key findings include:
- High Sensitivity: The system successfully detected approximately 90% of modified copies, even when 90% of the event types were removed from the stolen log.
- Outlier Detection Success: In cases of "Generation from Scratch" (manual re-typing of plagiarized code), the system flagged the logs as outliers nearly every time due to the absence of typical development rhythms.
- Robustness: Even when command execution counts were artificially altered by a factor of five, the system maintained a 40% detection rate for outliers, rising to 90% if half of the typical event types were missing.
Discussion and Conclusion
While process-based detection offers a powerful new tool against plagiarism, it introduces new challenges. Privacy is a primary concern, as logging every keystroke is inherently invasive. Furthermore, the system must account for "genius" outliers—highly efficient students whose clean workflows might mimic the lack of "messy" editing seen in plagiarized work.
Future work will focus on more complex sequence analysis, looking at the specific order of events rather than just their frequency, to counter sophisticated forgeries. Ultimately, these tools are intended to serve as a "first pass" for instructors, providing a visualization of the creation process that can be manually inspected to ensure pedagogical integrity.
Summary updated