A Comprehensive Review on Automated Grading Systems in STEM Using AI Techniques

L Tan, Shiyu Hu, Darren J. Yeo, Kang Hao CheongMathematics 2025 · DOI

TL;DR

This paper provides a comprehensive review of AI-powered automated grading systems (AGSs) in STEM education, detailing their algorithmic foundations, challenges, and future directions. It highlights how AI AGSs can significantly improve grading efficiency, consistency, and provide timely feedback, ultimately enhancing the learning experience for students.

What Problem Does This Paper Address?

Traditional manual grading, while offering personalized feedback, struggles with scalability, especially with increasing student numbers and complex assessments like programming assignments or essays. This leads to delayed feedback, increased teacher workload, and can even make grading for massive online courses infeasible. Furthermore, manual grading can be subjective and prone to bias. AI-powered AGSs aim to address these issues by automating the grading process, offering consistent, scalable, and immediate feedback across a wide range of assessment types in STEM fields, which often involve complex mathematical, scientific, and procedural elements.

Methods

The paper systematically reviews AI techniques used in AGSs, categorizing them into traditional machine learning (ML), deep learning (DL), and generative AI (GenAI) approaches.

  • Traditional ML & DL: These methods often rely on feature engineering, where raw data (text, code, diagrams) is transformed into meaningful features. Techniques like logistic regression, decision trees, support vector machines, convolutional neural networks (CNNs), and transformers are discussed. The paper emphasizes the challenges of feature engineering in STEM, which requires handling mathematical symbols, scientific formulas, and multimodal data.
  • GenAI: This section focuses on Large Language Models (LLMs) like GPT, which represent a paradigm shift. GenAI approaches bypass explicit feature engineering by directly understanding natural language. The paper highlights prompt engineering as a crucial aspect, where carefully crafted instructions guide the LLM to perform grading tasks. Techniques like zero-shot learning, Chain-of-Thought (CoT) reasoning, and few-shot learning are discussed as ways to improve LLM performance in grading.

The review also covers methodologies for evaluating AGSs, including metrics like precision, recall, F1-score, and Cohen's Kappa, as well as approaches to measure their impact on student learning and perceptions.

Datasets

The paper does not introduce new datasets but discusses how existing datasets are used for training and evaluating AGSs. It notes that a significant challenge is the scarcity of large, expertly-scored datasets, particularly for specialized STEM domains. Researchers are exploring methods like using GenAI models (e.g., GPT-4) for data augmentation to address this limitation.

Key Insights and Ideas

  • STEM's Unique Challenges: STEM assessments demand more than just keyword matching; they require evaluating mathematical accuracy, logical reasoning, procedural knowledge, and the interpretation of complex visual data. This necessitates sophisticated feature engineering and AI models.
  • GenAI's Transformative Potential: LLMs offer a more accessible and versatile approach to automated grading, moving beyond rule-based systems to nuanced understanding and generation of feedback, largely driven by effective prompt engineering.
  • The Human-in-the-Loop is Crucial: Despite AI's advancements, the paper consistently emphasizes the importance of human oversight, especially for high-stakes assessments, to ensure fairness and address potential AI errors.
  • Beyond Technical Skills: The future of AGSs will likely involve assessing soft skills like collaboration and communication, reflecting a broader trend towards holistic student evaluation.
  • Addressing Bias is Paramount: AI algorithms can inherit biases from their training data, leading to unfair outcomes for certain student groups. Diversifying training data and incorporating fairness-aware algorithms are critical.

Key Results

The paper synthesizes findings from numerous studies, rather than presenting its own experimental results. Key quantitative findings from cited works include:

  • AGSs can significantly reduce grading time (e.g., a 43% reduction reported for programming assignments).
  • Accuracy metrics vary widely depending on the algorithm and task, with some studies reporting agreement rates over 95% for specific tasks.
  • Student learning experience can be enhanced, with reports of improved performance and higher motivation due to immediate feedback. For instance, one study showed 89.7% of students felt immediate feedback improved their performance.
  • GenAI approaches, particularly with CoT reasoning, have shown significant improvements over basic prompting strategies.

Qualitatively, the paper highlights that AGSs improve grading consistency, reduce educator workload, and can provide valuable learning analytics.

Limitations and Future Work

Limitations:

  • Limited Expert-Scored Data: The scarcity of large, high-quality datasets for training AI models remains a significant challenge, especially for specialized STEM fields.
  • Student Trust and Perceived Fairness: Students may distrust AI graders, especially if they perceive errors or unfairness, leading to algorithm aversion.
  • Teacher Acceptance: Some educators exhibit algorithm aversion and fear of replacement, requiring professional development and clear communication about AI's role as an assistant.
  • Vulnerability to System Gaming: Students may exploit system weaknesses for higher grades, prioritizing score manipulation over genuine learning.
  • Over-reliance on Immediate Feedback: Students might become overly dependent on instant feedback, hindering deeper critical thinking and problem-solving skills.
  • Bias in Algorithmic Assessment: AGSs can perpetuate biases present in training data, unfairly penalizing certain student groups.
  • Institutional Readiness: Implementing AGSs requires robust infrastructure, IT support, and a willingness to adapt educational practices.

Future Work:

  • Explainable AI (XAI): Developing more transparent AI models to build trust and understanding.
  • Multimodal Assessment Systems: Integrating various data types (text, code, images, video, audio) for more comprehensive evaluations.
  • Virtual Reality (VR) Integration: Using VR for assessing practical skills and complex problem-solving in simulated environments.
  • Adaptive Learning Platforms (ALPs): Enhancing ALPs with AI for personalized learning trajectories and feedback.
  • Assessing Soft Skills: Developing AGSs to evaluate skills like collaboration, communication, and critical thinking.
  • Addressing Bias and Fairness: Continued research into developing bias-mitigation strategies and ensuring equitable assessment for all students.

Summary updated

Abstract

This paper presents a comprehensive analysis of artificial intelligence-powered automated grading systems (AI AGSs) in STEM education, systematically examining their algorithmic foundations, mathematical modeling approaches, and quantitative evaluation methodologies. AI AGSs enhance grading efficiency by providing large-scale, instant feedback and reducing educators’ workloads. Compared to traditional manual grading, these systems improve consistency and scalability, supporting a wide range of assessment types, from programming assignments to open-ended responses. This paper provides a structured taxonomy of AI techniques including logistic regression, decision trees, support vector machines, convolutional neural networks, transformers, and generative models, analyzing their mathematical formulations and performance characteristics. It further examines critical challenges, such as user trust issues, potential biases, and students’ over-reliance on automated feedback, alongside quantitative evaluation using precision, recall, F1-score, and Cohen’s Kappa metrics. The analysis includes feature engineering strategies for diverse educational data types and prompt engineering methodologies for large language models. Lastly, we highlight emerging trends, including explainable AI and multimodal assessment systems, offering educators and researchers a mathematical foundation for understanding and implementing AI AGSs into educational practices.
BibTeX
@inproceedings{tan2025comprehensive,
  author = {L Tan and Shiyu Hu and Darren J. Yeo and Kang Hao Cheong},
  title = {{A Comprehensive Review on Automated Grading Systems in STEM Using AI Techniques}},
  booktitle = {Mathematics},
  year = {2025},
  doi = {10.3390/math13172828},
}
Figure