Is your Statement Purposeless? Predicting Computer Science Graduation Admission Acceptance based on Statement Of Purpose

Diptesh Kanojia, Nikhil Wani, Pushpak Bhattacharyya

Introduction

The graduate school admission process, particularly for competitive Computer Science (CS) programs, is often perceived as a "black box." While quantitative metrics such as GPA and GRE scores provide clear benchmarks, the Statement of Purpose (SOP) remains a subjective and high-stakes component of the application. Applicants often lack quantitative tools to evaluate their essays, relying instead on vague university guidelines or expensive consultants. This research investigates a fundamental question: Can the information within an SOP alone be used to predict the outcome of a candidate's application?

Methodology

The researchers framed the problem as a binary classification task (Accept vs. Reject). The proposed system architecture, the Classification Module, extracts features from an applicant's SOP and compares them against a curated corpus of successful statements to generate a prediction.

The Classification Module architecture illustrating the flow from applicant SOP to feature extraction and final classifier prediction.

Feature Engineering

Three distinct feature sets were developed to capture different aspects of the text:

  1. Textual Features (T): 14 linguistic markers including Part-of-Speech (PoS) ratios, discourse connectors, readability scores (e.g., Flesch Reading Ease), and basic length metrics.
  2. Word Embedding Features (WE): 300-dimensional vectors derived from pre-trained Google News corpora to capture deep semantic meaning.
  3. Similarity & Error Features (SE): This set included Cosine similarity against a "gold standard" corpus of successful SOPs, GloVe-based sentence similarity, and counts of spelling or out-of-vocabulary errors.

Several algorithms were evaluated, including Support Vector Machines (SVM) with an RBF kernel, Logistic Regression (LR), Random Forest Decision Trees (RFDT), and Feed Forward Neural Networks (FFNN).

Dataset

The study utilized a custom dataset of 50 manually verified SOPs, split evenly between 25 accepted and 25 rejected samples. Accepted SOPs were sourced from "Elite" universities (acceptance rates ≤ 15%). Data was gathered from personal networks, public personal websites, and admission consultancy blogs. Despite the modest sample size, the authors emphasized the rigor of the manual verification process.

Results and Analysis

The experimental results demonstrated that the choice of features is more critical than the complexity of the model. An ablation study was conducted to determine the impact of different feature combinations across various validation methods (N-fold cross-validation and a 50% split).

Performance comparison (F-score) of individual and combined feature sets across different validation splits.

The most significant finding was that the combination of Similarity and Word Embedding features (SE + WE) yielded the highest performance, achieving an F-score of 92%. In contrast, traditional textual features (T) performed poorly on their own, often yielding results near chance (40–54%). This suggests that successful SOPs share a specific semantic "fingerprint" or "vibe" that is more predictive than standard metrics of "good writing" like grammar or readability.

Among the classifiers, the SVM outperformed deep learning approaches. While the Multilayer Perceptron reached an 82% average F-score, the Feed Forward Neural Network struggled significantly (36–60%), likely due to the small size of the dataset which favors well-regularized traditional models over data-hungry neural architectures.

Discussion and Future Work

The study provides a proof-of-concept that semantic alignment with successful essays is a powerful predictor of admission. However, the researchers acknowledge limitations, primarily the small dataset size. Furthermore, a truly holistic admission model would need to incorporate other critical components such as GRE scores, GPA, and Letters of Recommendation (LORs).

Future directions include integrating Recurrent Neural Networks (RNNs) to better capture sequential text patterns and deploying the tool as a Free and Open Source (FOSS) web service. Such a tool could democratize the application process by helping students evaluate their own statements before submission.

Summary updated

BibTeX
@inproceedings{kanojia2017is,
  author = {Diptesh Kanojia and Nikhil Wani and Pushpak Bhattacharyya},
  title = {{Is your Statement Purposeless? Predicting Computer Science Graduation Admission Acceptance based on Statement Of Purpose}},
  booktitle = {ICON},
  year = {2017},
  url = {https://openreview.net/forum?id=i7hCvZm6Jy5},
}
Figure