Abstract
Code Author Identification (AI) is the task of attributing source code to its original author based on stylistic features. This paper evaluates two primary strategies: a standard machine learning classification pipeline and a simpler, retrieval-inspired ranking approach. While classification using Random Forest achieved respectable results, a ranking method based on long character n-grams (15-grams) proved superior. This approach achieved 91.57% accuracy on the AI-SOCO dataset, placing 5th in the FIRE2020 competition and demonstrating that simple stylistic "fingerprints" can rival complex neural models.
Introduction
Identifying the author of source code has significant implications for software engineering and cybersecurity. Applications range from detecting plagiarism in academic or professional settings to forensic analysis of malware and resolving authorship in legacy systems where version control data is absent. However, code authorship is uniquely challenging because the rigid syntax of programming languages limits the stylistic freedom available to authors compared to natural language.
Methodology
The researchers explored two distinct paradigms for authorship attribution:
1. Classification-based Approach
This approach follows a standard machine learning workflow. Source code snippets are converted into numerical feature vectors using TF-IDF (Term Frequency-Inverse Document Frequency). Several classifiers were evaluated, including Support Vector Machines (SVM) and K-Nearest Neighbors (KNN). Among these, the Random Forest classifier emerged as the most effective baseline for the classification task.
2. Ranking-based Approach
The ranking-based method treats authorship identification as an information retrieval problem. For each author in the training set, all available code snippets are concatenated into a single representative document. To identify the author of a new, anonymous snippet, the system calculates its similarity to each author's aggregate document.
The similarity metric is based on the frequency of character-based n-grams. The team experimented with varying lengths of $n$ to determine the optimal granularity for capturing individual coding styles.
Experimental Setup
The study utilized the AI-SOCO (Author Identification of SOurce COde) dataset from the FIRE2020 competition.
- Data Source: 100,000 C++ source code files from the Codeforces platform.
- Scale: 1,000 unique authors, each represented by 100 samples.
- Data Split: 50,000 samples for training, 25,000 for validation, and 25,000 for testing.
Results and Discussion
The experimental results revealed a clear advantage for the ranking-based approach over traditional classification.
The Power of Long N-grams
A critical finding was the relationship between n-gram length and identification accuracy. Short n-grams (4-6 characters) performed poorly because they primarily capture common language keywords (e.g., for, while, int) shared by all programmers. In contrast, longer sequences (15-20 characters) effectively capture idiosyncratic habits, such as specific variable naming conventions, indentation patterns, and unique logical structures. As shown in the experimental data, the 15-gram configuration achieved the highest validation accuracy of 0.9196.
Competition Performance
In the final evaluation on the hidden test set, the 15-gram ranking model achieved an accuracy of 91.57%. This performance secured the 5th place overall in the AI-SOCO competition. Notably, this relatively simple method performed nearly as well as the RoBERTa-based baseline (0.9288), despite requiring significantly less computational overhead for training and inference.
Conclusion
This research demonstrates that for code authorship identification, a retrieval-based ranking strategy using long character n-grams is both highly effective and computationally efficient. While powerful, the current approach is limited to one-dimensional character sequences. Future work will explore incorporating structural features from Abstract Syntax Trees (ASTs), more sophisticated "learning to rank" algorithms, and global features like cyclomatic complexity to further enhance attribution accuracy.
Summary updated