Abstract
This paper introduces a deep learning framework (AttLSTM-CNN) designed to move beyond subjective "star ratings" in education by automatically analyzing the sentiment of student feedback. By combining attention mechanisms with domain-specific sentiment dictionaries, the authors achieved a 21.9% improvement in evaluation quality, providing teachers with actionable insights into student confusion and engagement.
Introduction
In the realm of Chinese grammar instruction, student evaluations are a critical feedback loop. However, traditional evaluation methods suffer from high subjectivity; a student's mood, personal preferences, or a single difficult lesson can skew a numerical score, making it a poor metric for actual teaching quality. Furthermore, simple scores don't capture the why behind the feedback. A teacher might know a lesson was poorly received but won't know if the issue was the "classroom atmosphere" or the specific complexity of "separable words." This research aims to bridge this gap by using Natural Language Processing (NLP) to objectively quantify the emotional and attitudinal tendencies hidden within unstructured text comments.
Methodology
The authors propose a hybrid deep learning architecture called AttLSTM-CNN. The system follows a structured pipeline from data acquisition to visualization of results.
The AttLSTM-CNN Pipeline
- Preprocessing & Domain Adaptation: Using the Jieba segmenter, the system processes Chinese text, removes stop words, and normalizes colloquialisms. An iterative screening strategy was used to build a domain-specific sentiment dictionary of nearly 10,000 entries, ensuring the model understands "teaching rhythm" differently than it might in a general context.
- The AttLSTM Layer: An Attention-based Long Short-Term Memory network encodes the text. The LSTM captures long-distance dependencies between words, while the Attention mechanism assigns higher weights to "emotionally charged" words that define the sentence's sentiment.
- The CNN Layer: A Convolutional Neural Network is used for feature extraction, identifying local patterns and reducing the dimensionality of the word vector matrix through max-pooling.
- Classification: While the model can perform binary classification, it is extended to a multi-label framework (confused, satisfied, frustrated, neutral) to capture complex emotional states like "satisfied but confused."
Key Insights and Results
The study utilized a combination of established linguistic resources and a real-world corpus of 2,000 student evaluation texts. Key findings include:
- Performance: The AttLSTM-CNN model achieved an F-measure of 0.76, significantly outperforming traditional SVM classifiers (0.58). It also outperformed BERT-base-Chinese in accuracy while being only 1/5 the size in terms of parameters, making it suitable for lightweight deployment.
- Evaluation Quality: The implementation of this technology improved the overall quality and objectivity of the teaching evaluation system by approximately 21.9%.
- Actionable Analytics: In one case study, the model identified that "confusion" accounted for 35% of comments regarding "separable words." By adjusting the teaching strategy to include visual aids, the teacher reduced negative comments to 12%.
- Domain Sensitivity: The research found that 23% of sentiment words in general public dictionaries rarely appear in educational contexts, highlighting the necessity of the custom domain-adapted dictionary.
Limitations and Future Work
Despite its success, the architecture faces challenges with very long texts (over 512 words), where accuracy drops by 12% due to attention decay. Additionally, the sentiment dictionary requires manual maintenance to keep up with evolving internet slang. Future research aims to incorporate multimodal data, such as classroom video recordings and eye-tracking data, to create a more holistic analysis framework for emotional evaluation.
Summary updated