Abstract
Data privacy remains a critical concern in educational research, requiring strict adherence to ethical standards and regulatory protocols. While traditional approaches rely on anonymization and centralized data collection, they often expose raw student data to security vulnerabilities and impose substantial logistical overhead. In this study, we propose a federated learning (FL) framework for automated scoring of educational assessments that eliminates the need to share sensitive data across institutions. Our approach leverages parameter-efficient fine-tuning of large language models (LLMs) with Low-Rank Adaptation (LoRA), enabling each client (school) to train locally while sharing only optimized model updates. To address data heterogeneity, we implement an adaptive weighted aggregation strategy that considers both client performance and data volume. We benchmark our model against two state-of-the-art FL methods and a centralized learning baseline using NGSS-aligned multi-label science assessment data from nine middle schools. Results show that our model achieves the highest accuracy (94.5%) among FL approaches, and performs within 0.5-1.0 percentage points of the centralized model on these metrics. Additionally, it achieves comparable rubric-level scoring accuracy, with only a 1.3% difference in rubric match and a lower score deviation (MAE), highlighting its effectiveness in preserving both prediction quality and interpretability.
Introduction
In educational research, AI-driven automated scoring provides rapid, scalable feedback. However, traditional training requires "centralized data aggregation"—pooling student responses from various schools into a single database. This approach faces significant hurdles due to strict privacy laws, such as FERPA in the US, which often prohibit moving sensitive student data across district lines.
Furthermore, educational data is inherently heterogeneous. Different schools exhibit varying demographics, curricula, and grading standards. A "one-size-fits-all" centralized model often struggles to generalize across these diverse contexts. This research introduces a Federated Learning (FL) framework that allows institutions to collaborate on model training without compromising student privacy or performance.
Methodology
The proposed framework keeps data local to each school (the "client"), sharing only model updates rather than raw data.
System Architecture
The architecture consists of multiple clients and a central server connected via a secure communication channel. Key technical components include:
- Parameter-Efficient Fine-Tuning (PEFT) via LoRA: Instead of updating all weights of a Large Language Model (LLM), the framework uses Low-Rank Adaptation (LoRA) with a rank of 8 on a frozen TinyLLaMA base. This reduces network traffic and minimizes the risk of data leakage through gradient updates.
- Adaptive Weighted Aggregation: Unlike standard FL, which weights updates by sample size alone, this framework incorporates local model performance (validation loss). This ensures high-quality updates from better-performing clients have a stronger influence on the global model.
- Security and Privacy: To prevent reverse-engineering of student data, calibrated Gaussian noise (Differential Privacy) is added to updates. Communication is secured using gRPC with mutual TLS (mTLS).
Dataset and Evaluation
The study utilized real-world data from the PASTA project, involving approximately 1,200 middle school students across nine diverse school systems. The dataset includes nine multi-label science tasks aligned with the Next Generation Science Standards (NGSS), evaluated using a structured five-dimensional rubric.
Results and Discussion
The proposed FL model was benchmarked against a Centralized Learning (CL) baseline and two state-of-the-art (SOTA) FL methods.
The results demonstrate that the proposed framework effectively closes the "federated gap":
- Accuracy: The model achieved 94.5% accuracy, outperforming SOTA FL methods (90–92%) and performing competitively with the centralized model.
- Rubric Alignment: It achieved an 88.2% rubric-level match, showing high alignment with human raters.
- Error Reduction: The model recorded the lowest Mean Absolute Error (MAE) of 0.34, significantly outperforming SOTA FL baselines (0.52–0.60).
Ablation studies confirmed the importance of the adaptive weighting strategy; removing it led to a 2.8% drop in accuracy and a 4.1% drop in rubric match, proving the necessity of the novel aggregation strategy.
Conclusion
This research demonstrates that Federated Learning, combined with parameter-efficient fine-tuning and adaptive aggregation, can achieve high-performance automated scoring while strictly adhering to educational privacy regulations. While current results are focused on middle school science, future work will explore scaling to more subjective subjects and larger federated networks, as well as optimizing the trade-off between Differential Privacy noise and model accuracy.
Summary updated