A Review of the Trends and Challenges in Adopting Natural Language Processing Methods for Education Feedback Analysis

Thanveer Basha Shaik, Xiaohui Tao, Yan Li, Christopher Dann, Jacquie McDonald, Petrea Redmond, Linda Galligan

TL;DR

This paper provides a comprehensive survey of how Natural Language Processing (NLP) is being used to automate the analysis of student feedback. It maps the transition from traditional statistical methods to modern deep learning architectures like BERT, while identifying the "hard problems" of educational data, such as sarcasm, domain-specific jargon, and data imbalance.

What Problem Does This Paper Address?

In the modern educational landscape—ranging from traditional universities to Massive Open Online Courses (MOOCs)—student feedback is a primary driver for institutional improvement. However, qualitative feedback (textual comments) is notoriously difficult to scale. Manually reading and categorizing thousands of student opinions is time-consuming, subjective, and resource-intensive.

The authors tackle the challenge of "opinion mining" in education. They explore how NLP can bridge the gap between raw student text and actionable insights regarding teaching practices, curriculum design, and learning environments. The core challenge lies in the complexity of human language: students use sarcasm, emoticons, and highly specific academic terminology that standard "off-the-shelf" sentiment analyzers often fail to capture.

Methods

The paper categorizes the NLP pipeline for education into several key methodological stages:

  • Feature Extraction: The authors compare traditional methods like Bag of Words (BoW) and TF-IDF (which focus on word frequency) against modern Word Embeddings (Word2Vec, GloVe) and Transformers (BERT). They highlight BERT’s ability to generate dynamic, context-aware representations as a significant leap forward.
  • Feature Selection: To reduce dimensionality and improve model performance, the paper reviews Filter (statistical ranking), Wrapper (iterative selection via classifiers), and Embedded (regularization-based) methods.
  • Topic Modelling: The review focuses heavily on Latent Dirichlet Allocation (LDA) as the dominant unsupervised method for discovering hidden themes in student comments without prior labeling.
  • Text Evaluation: The authors distinguish between Extractive Summarization (picking the most important sentences) and Abstractive Summarization (generating new text that paraphrases the original content), noting that deep learning (LSTM-CNN) is pushing the boundaries of the latter.

Datasets

As a review article, this paper does not introduce a single new dataset but synthesizes findings from various sources used in the field:

  • MOOC Datasets: Feedback from platforms like Coursera and edX.
  • Institutional Surveys: Internal student satisfaction data from universities.
  • Social Media/Technical Forums: Data from Twitter, Reddit, and StackOverflow (used to train models for technical education contexts).
  • Benchmark Comparisons: The paper references the use of BBC news datasets and Amazon product reviews as benchmarks to test how well models transfer to the educational domain.

Key Insights and Ideas

  • The Contextual Shift: The most significant trend is the move away from "word-counting" to "context-understanding." Models like BERT are preferred because they can distinguish between the same word used in different academic contexts.
  • Aspect-Based Sentiment Analysis (ABSA): The authors argue that knowing a student is "unhappy" isn't enough. ABSA is critical because it identifies what they are unhappy about—be it the "lab equipment" (the aspect) vs. the "lecture clarity."
  • The Sarcasm Barrier: Sarcasm remains a "frontier" challenge. The paper notes that traditional classifiers often misread sarcastic praise as positive, necessitating multi-modal approaches that look at "sentiment flips" and context.
  • The Role of Emojis: In modern feedback, emoticons carry heavy semantic weight. The research community is increasingly using Bi-LSTM and CNN models to process emojis as distinct features that convey emotional nuance.

Key Results

The paper summarizes quantitative successes from the literature it reviews:

  • Accuracy Benchmarks: Deep learning models (CNN, LSTM) consistently achieve higher accuracy (often 86% to 94%) in sentiment classification compared to traditional Naive Bayes or SVM models.
  • Summarization Gains: Hybrid summarization models have shown a 4.4% increase in ROUGE-1 scores over older state-of-the-art systems.
  • Efficiency: Using BERT for keyword extraction allowed researchers to reduce feedback text by 59% while maintaining high predictive power for course ratings.
  • Tooling: Python is identified as the leading language for these implementations due to robust libraries like spaCy, NLTK, and Hugging Face (Transformers).

Limitations and Future Work

The authors identify several "bottlenecks" that the field has yet to solve:

  • Data Scarcity & Imbalance: There is a lack of massive, high-quality, labeled datasets specifically for education. Furthermore, feedback is often imbalanced (e.g., mostly very positive or very negative), which biases models.
  • The "Black Box" Problem: Deep learning models lack interpretability. In education, stakeholders need to know why a model flagged a certain comment, especially when it impacts teacher evaluations.
  • Privacy Concerns: Extracting insights from student feedback must be balanced with anonymity and ethical data usage.
  • Pedagogical Integration: Most current research focuses on the technical performance of NLP rather than how these tools actually improve pedagogical outcomes in the long term.

Summary updated

Abstract

Artificial Intelligence (AI) is a fast-growing area of study that stretching its presence to many business and research domains. Machine learning, deep learning, and natural language processing (NLP) are subsets of AI to tackle different areas of data processing and modelling. This review article presents an overview of AI’s impact on education outlining with current opportunities. In the education domain, student feedback data is crucial to uncover the merits and demerits of existing services provided to students. AI can assist in identifying the areas of improvement in educational infrastructure, learning management systems, teaching practices and study environment. NLP techniques play a vital role in analyzing student feedback in textual format. This research focuses on existing NLP methodologies and applications that could be adapted to educational domain applications like sentiment annotations, entity annotations, text summarization, and topic modelling. Trends and challenges in adopting NLP in education were reviewed and explored. Context-based challenges in NLP like sarcasm, domain-specific language, ambiguity, and aspect-based sentiment analysis are explained with existing methodologies to overcome them. Research community approaches to extract the semantic meaning of emoticons and special characters in feedback which conveys user opinion and challenges in adopting NLP in education are explored.
BibTeX
@inproceedings{shaik2022review,
  author = {Thanveer Basha Shaik and Xiaohui Tao and Yan Li and Christopher Dann and Jacquie McDonald and Petrea Redmond and Linda Galligan},
  title = {{A Review of the Trends and Challenges in Adopting Natural Language Processing Methods for Education Feedback Analysis}},
  booktitle = {IEEE Access},
  year = {2022},
  url = {https://openreview.net/forum?id=R1unjGbbtV},
}

Loading related content...

Figure