A Study of Educational Data Mining: Evidence from a Thai University

Ruangsak Trakunphutthirak, Yen Cheung, Vincent C. S. Lee

Introduction

Identifying students at risk of academic failure is a critical Key Performance Indicator (KPI) for higher education institutions. Traditional methods often rely on manual surveys or Learning Management System (LMS) logs. However, surveys are resource-intensive and prone to participation bias, while LMS data is often siloed within specific courses, failing to provide a holistic view of student behavior. This study explores an alternative, non-intrusive data source: university firewall logs. By capturing all campus internet requests, these logs offer an automated and comprehensive perspective on student activity across their entire academic environment.

Methodology

The researchers implemented an Educational Data Mining (EDM) framework to process approximately 70GB of raw firewall logs daily. The approach utilized a "schema on read" strategy to manage semi-structured data efficiently. Feature engineering focused on two primary domains: Duration (total time spent) and Frequency (number of access hits).

The study involved 294 science students living off-campus. Data was categorized into two datasets:

  • Dataset-1 (CAT): 24 broad web categories (e.g., social networking, search engines).
  • Dataset-2 (APP): 147 specific application types (e.g., Facebook-base, YouTube-base).

Students were classified as "at-risk" if their GPA fell below 2.4. Five machine learning models were evaluated: Naive Bayes, Logistic Regression, Neural Networks, Decision Trees (J48), and Random Forest.

Behavioral Insights

Analysis revealed that the duration of internet sessions is a significantly stronger predictor of academic success than the frequency of access. Specific application data (APP) also proved more descriptive than broad categories (CAT).

Average time spent per hour by student risk category.

A notable finding was the "Gaming Paradox": successful students spent more time on online games (avg. 6.48 min/session) than at-risk students (avg. 1.86 min/session), suggesting that high-performing students may use gaming for structured stress relief. Conversely, at-risk students generally spent more time online per hour (2.72 minutes vs. 2.28 minutes) during the first week of the semester, potentially indicating early procrastination.

Key Results

The Random Forest algorithm emerged as the most effective model, consistently outperforming others across all datasets with an average accuracy of 0.778.

Comparison of algorithm accuracy across different datasets.

The highest predictive accuracy (0.779) was achieved using the APP-Duration dataset. Further optimization showed that a weighted combination of 90% Application data and 10% Category data provided the most robust identification of at-risk students.

Conclusion and Future Work

While firewall logs provide a powerful "invisible" data source for predicting student outcomes, the study notes limitations such as the small sample size and the short one-week data collection window. Future research aims to extend the analysis over full academic years and explore Deep Learning models to capture more complex behavioral patterns. The ultimate objective is to integrate these predictive insights into real-time academic intervention systems.

Summary updated

Abstract

Educational data mining provides a way to predict student academic performance. A psychometric factor like time management is one of the major issues affecting Thai students’ academic performance. Current data sources used to predict students’ performance are limited to the manual collection of data or data from a single unit of study which cannot be generalised to indicate overall academic performance. This study uses an additional data source from a university log file to predict academic performance. It investigates the browsing categories and the Internet access activities of students with respect to their time management during their studies. A single source of data is insufficient to identify those students who are at-risk of failing in their academic studies. Furthermore, there is a paucity of recent empirical studies in this area to provide insights into the relationship between students’ academic performance and their Internet access activities. To contribute to this area of research, we employed two datasets such as web-browsing categories and Internet access activity types to select the best outcomes, and compared different weights in the time and frequency domains. We found that the random forest technique provides the best outcome in these datasets to identify those students who are at-risk of failure. We also found that data from their Internet access activities reveals more accurate outcomes than data from browsing categories alone. The combination of two datasets reveals a better picture of students’ Internet usage and thus identifies students who are academically at-risk of failure. Further work involves collecting more Internet access log file data, analysing it over a longer period and relating the period of data collection with events during the academic year.
BibTeX
@inproceedings{trakunphutthirak2019study,
  author = {Ruangsak Trakunphutthirak and Yen Cheung and Vincent C. S. Lee},
  title = {{A Study of Educational Data Mining: Evidence from a Thai University}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.3301734},
}
Figure