Student sentiment Analysis Using Classification With Feature Extraction Techniques

Disha ShrivastavaarXiv 2021 · arXiv

TL;DR

This paper tackles the challenge of automatically analyzing student feedback from web-based learning systems to gauge satisfaction. By applying and comparing various machine learning classifiers with different feature extraction techniques, the authors demonstrate that SVM, particularly with the BoW feature extraction method, offers the best performance in classifying sentiment from student reviews.

What Problem Does This Paper Address?

Web-based learning systems (WBLS) are increasingly popular, but ensuring student satisfaction and understanding their learning experience remains a challenge. Traditional feedback mechanisms often rely on rating scales, which can be ambiguous (e.g., a neutral rating might accompany negative comments). This paper addresses the need for a more nuanced approach to analyze student feedback, specifically focusing on extracting sentiment from textual reviews to better understand student satisfaction with WBLS. The goal is to move beyond simple ratings to a deeper understanding of student opinions, which can inform improvements to online learning platforms.

Methods

The core of this research involves applying and comparing several machine learning (ML) classification algorithms for sentiment analysis. The authors explore:

  • Feature Extraction Techniques (FETs):
    • Bag of Words (BoW): A simple method that represents text as a multiset of words, counting the occurrences of each word.
  • Term Frequency-Inverse Document Frequency (TF-IDF): A more sophisticated method that assigns weights to words based on their frequency within a document and their rarity across the entire corpus, reflecting a word's importance.
  • Classification Algorithms:
    • Logistic Regression (LR): A statistical model used to predict the probability of an event.
  • Support Vector Machine (SVM): A powerful algorithm for classification that finds an optimal hyperplane to separate data points.
  • Naive Bayes (NB): A probabilistic classifier based on Bayes' theorem, assuming independence between features.
  • Decision Tree (DT): A tree-like structure used for decision-making, where internal nodes represent features and leaf nodes represent class labels.

The study also incorporates text preprocessing steps, including removing numbers and punctuation, converting text to lowercase, tokenization, removing stop words, lemmatization, and spelling correction, to clean the data before feature extraction. The performance of these models is evaluated using two data partitioning strategies (70/30 and 80/20 train/test splits) and assessed using metrics like Accuracy, Sensitivity, Specificity, and F1-Score.

Datasets

The primary dataset used in this study is the Student Feedback (SFB) dataset.

  • Source: Collected from a web portal named http://elearningit.in.
  • Content: Student reviews and feedback on courses, content, and the Learning Management System (LMS).
  • Collection Period: January 2020 to June 2020.
  • Size: Originally 549 comments.
  • Classes: Divided into two labels: Positive comments (class label -1) and Negative comments (class label 0).
  • Variations: The dataset is used in both its raw (Unclean SFB - USFB) and pre-processed (Normalized SFB - NSFB) forms.

Key Insights and Ideas

  • The Importance of Preprocessing: Cleaning the text data (removing noise like punctuation, stop words, etc.) significantly impacts the performance of sentiment analysis models. The normalized dataset (NSFB) generally yielded better results than the uncleaned one (USFB).
  • Feature Extraction Matters: Both BoW and TF-IDF are effective feature extraction techniques, but their performance can vary depending on the classifier and the dataset. BoW, despite its simplicity, proved to be highly effective in conjunction with certain classifiers.
  • SVM's Dominance: The Support Vector Machine (SVM) classifier consistently outperformed other models (LR, NB, DT) across various configurations, especially when combined with the Bag of Words (BoW) feature extraction method.
  • Visualizing Sentiment: Word clouds provide an intuitive way to visualize the most frequent words in student feedback, offering a quick understanding of common themes and sentiments. The word cloud in this study indicated a generally positive sentiment, with words like "thanks," "good," and "great" appearing frequently.

Key Results

The study evaluated the performance of four ML classifiers (LR, NB, SVM, DT) using two feature extraction techniques (BoW, TF-IDF) and two data partitioning methods (DPT1: 70/30, DPT2: 80/20).

  • Highest Accuracy: The SVM classifier achieved the highest accuracy of 87.27% using the BoW feature extraction method with the normalized dataset (NSFB) and the 70/30 data partition (DPT1).
  • TF-IDF Performance: With TF-IDF, the DT classifier achieved the highest F1-Score of 67.89% on the normalized dataset (NSFB) for DPT2.
  • Overall Trend: SVM generally showed superior performance, particularly with the BoW feature. The normalized dataset consistently performed better than the uncleaned dataset.

Limitations and Future Work

  • Dataset Size and Scope: The study used a dataset of 549 comments collected over six months from a specific web portal. This might limit the generalizability of the findings to other WBLS or larger, more diverse datasets.
  • Feature Extraction Limitations: While BoW and TF-IDF are effective, more advanced techniques like Word2Vec or BERT could potentially capture deeper semantic relationships and improve performance further.
  • Future Work Suggestions: The authors implicitly suggest exploring more advanced feature extraction techniques and potentially larger, more diverse datasets to further enhance sentiment analysis accuracy in WBLS. Investigating the impact of different preprocessing steps and exploring ensemble methods could also be valuable future directions.

Summary updated

BibTeX
@misc{shrivastava2021student,
  author = {Disha Shrivastava},
  title = {{Student sentiment Analysis Using Classification With Feature Extraction Techniques}},
  year = {2021},
  eprint = {2102.05439},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2102.05439},
}
Figure