Abstract
Access to higher education is critical for minority populations and emergent bilingual students. However, the language used by higher education institutions to communicate with prospective students is often too complex; concretely, many institutions in the US publish admissions application instructions far above the average reading level of a typical high school graduate, often near the 13th or 14th grade level. This leads to an unnecessary barrier between students and access to higher education. This work aims to tackle this challenge via text simplification. We present PSAT (Professionally Simplified Admissions Texts), a dataset with 112 admissions instructions randomly selected from higher education institutions across the US. These texts are then professionally simplified, and verified and accepted by subject-matter experts who are full-time employees in admissions offices at various institutions. Additionally, PSAT comes with manual alignments of 1,883 original-simplified sentence pairs. The result is a first-of-its-kind corpus for the evaluation and fine-tuning of text simplification systems in a high-stakes genre distinct from existing simplification resources.
Introduction: The Linguistic Gatekeeper
For many prospective students—particularly first-generation applicants, low-income individuals, and English language learners—the path to college is often blocked by a "linguistic gatekeeper." Research indicates that US college admissions instructions are frequently written at a 13th or 14th-grade reading level, significantly exceeding the comprehension of the average high school graduate. When the instructions for applying to an institution are more complex than the education the student has yet received, the process becomes exclusionary.
While Automatic Text Simplification (ATS) offers a potential technological solution to this barrier, existing models are primarily trained on general-domain datasets such as Wikipedia or news articles. These models often fail in "high-stakes" environments where losing a specific detail—such as the distinction between an "official" and "unofficial" transcript—can result in a rejected application. To address this gap, we introduce the Professionally Simplified Admissions Texts (PSAT) corpus, a specialized, professionally verified dataset designed to train and evaluate models on admissions-specific text.
The PSAT Corpus Development
The creation of the PSAT corpus followed a rigorous, expert-led pipeline to ensure both linguistic accessibility and administrative accuracy:
- Extraction: Raw instructions were collected from 112 randomly sampled US postsecondary institutions using the Integrated Postsecondary Education Data System (IPEDS) database.
- Professional Simplification: A domain expert with a doctorate in education and over a decade of admissions experience manually simplified the texts. The focus was on reducing syntactic complexity, increasing lexical cohesion, and providing necessary elaboration on technical jargon.
- Expert Verification: To ensure no critical information was lost, ten subject-matter experts (SMEs)—including full-time admissions officers from diverse institutions ranging from community colleges to private universities—reviewed the simplifications in real-time. This step ensured that the simplified text remained legally and procedurally accurate.
- Alignment: The authors manually aligned 1,883 original-to-simplified sentence pairs to facilitate machine learning applications.
Experimental Benchmarking
To establish a baseline for the dataset, we evaluated two primary models: ACCESS (a controllable simplification model) and T5 (Text-to-Text Transfer Transformer). These models were tested in zero-shot settings and after fine-tuning on the PSAT data. We compared these results against standard simplification datasets like Wikilarge and Newsela to measure cross-domain performance.
The Challenge of High-Stakes Nuance
Our findings highlight that admissions text requires extreme precision. SMEs noted that seemingly simple word substitutions can change the meaning of a requirement. For example, changing "submit" to "complete" was often flagged as unacceptable because "submit" implies a specific, final action by the applicant toward the institution.
Furthermore, general-domain models often deleted "redundant" phrases that were actually critical. If a college accepts both the Common App and the Coalition App, a model trained on Wikipedia might delete one to improve brevity, inadvertently providing false or incomplete instructions to the student.
Key Results and Insights
- Reading Level Reduction: Professional simplification successfully reduced the average reading level of the instructions from Grade 13.3 to Grade 9.8, moving the text from "college-level" to a level accessible to the majority of high school graduates.
- Performance Gains via Fine-Tuning: Fine-tuning on PSAT led to significant improvements in automatic metrics. The T5-wiki-ft model (pre-trained on Wikipedia and fine-tuned on PSAT) achieved the highest BERTScore (0.923) and BLEU (0.239).
- Metric Discrepancies: The ACCESS model achieved the highest SARI score (0.271). However, manual inspection suggested that while SARI rewards word changes (additions and deletions), the T5 model's outputs were often more accurate in preserving the specific "professional voice" and factual integrity required for admissions.
- The "None" Factor: In manual evaluations, human experts frequently chose "None" when asked to rank model outputs. This indicates that even fine-tuned models still struggle to perfectly capture the nuance required for high-stakes administrative communication.
Conclusion and Future Work
The PSAT corpus represents a significant step toward making higher education more accessible by providing a high-quality resource for high-stakes text simplification. While the current dataset is limited to 112 documents and a US-centric context, it provides a vital foundation for future research.
Future work will focus on expanding the corpus to include paragraph-level simplifications and exploring the integration of these automated tools into admissions offices. By helping practitioners communicate more clearly, we can begin to dismantle the linguistic barriers that prevent qualified students from pursuing their educational goals.
Summary updated