Abstract
In large-scale educational environments, instructors often utilize reflection prompts to monitor student comprehension. However, the volume of responses can be overwhelming. This paper presents a novel phrase-based summarization algorithm that extracts noun phrases, clusters them semantically using K-Medoids and Latent Semantic Analysis, and ranks them based on "student coverage." Evaluation on a dataset of undergraduate responses demonstrates that this approach significantly outperforms traditional sentence-based summarization methods like LexRank and MEAD in terms of ROUGE scores.
Introduction
Instructors in large STEM courses or MOOCs frequently use reflection prompts (e.g., "What was the most confusing part of today's lecture?") to identify student difficulties. Manually processing hundreds of these responses is an onerous task. Existing automated summarization tools often fail because student responses vary significantly in length and linguistic structure. Furthermore, standard algorithms prioritize centrality or diversity, whereas educational contexts require prioritizing "coverage"—the number of unique students mentioning a specific concept.
Methods
The proposed phrase summarization pipeline consists of three stages:
- Candidate Phrase Extraction: Using the Senna syntax parser, the system extracts Noun Phrases (NPs) to focus on core knowledge concepts rather than filler text.
- Phrase Clustering: To handle lexical variability (e.g., "bicycle parts" vs. "bike elements"), phrases are grouped using K-Medoids clustering. Semantic distance is calculated via Latent Semantic Analysis (LSA) trained on an educational corpus.
- Phrase Ranking: Importance is defined by "student coverage"—the count of unique students contributing to a cluster. Within each cluster, LexRank is used to select the most representative phrase for the final summary.
Experimental Setup
The system was evaluated using a dataset from 53 undergraduates in an introductory materials science course, covering 25 lectures and over 900 responses. Reference summaries were provided by Teaching Assistants (TAs) to establish a gold standard for both content and student coverage counts.
Results
The proposed method was compared against several baselines, including Keyphrase extraction (Maui), LexRank, and MEAD, using ROUGE metrics. The results indicate that the phrase-based approach achieves significantly higher F-measures across R-1, R-2, and R-SU4 metrics.
Statistical tests confirmed that the model outperformed LexRank in precision across all metrics. Furthermore, the use of LexRank for intra-cluster selection proved superior to simple medoid selection, effectively reducing redundancy while maintaining representativeness. These findings suggest that semantic clustering combined with coverage-based ranking aligns more closely with human pedagogical summarization than standard graph-based centrality alone.
Conclusion
Phrase-based summarization offers a more effective way to digest student feedback, particularly for mobile interfaces where screen space is limited. By focusing on semantic coverage rather than just lexical overlap, the system provides instructors with a clearer picture of classroom needs. Future work will explore including broader linguistic constituents and conducting extrinsic evaluations to measure the impact on actual teaching outcomes and student learning.
Summary updated