Abstract
Automated scoring of open-ended student responses typically requires deploying individual large-scale neural networks for each assessment task, leading to significant storage and computational overhead. This paper introduces GW-SMM (Gromov-Wasserstein Scoring Model Merging), a novel framework that leverages Gromov-Wasserstein distance to align internal feature distributions of specialized models for efficient merging. By quantifying structural similarities in how models represent student thinking, GW-SMM enables the creation of unified multi-task models that reduce storage requirements by up to 3x while maintaining high scoring accuracy. Experimental results on physical science assessment data demonstrate that GW-SMM outperforms both human-expert and LLM-based merging strategies.
Introduction
In educational technology, deep learning models like BERT have become indispensable for scoring complex, open-ended student responses. However, a major bottleneck exists: the "one model per task" paradigm. Deploying dozens of 450MB models for a single assessment is impractical for storage and maintenance. While training a single multi-task model from scratch is an alternative, it often suffers from performance degradation compared to specialized counterparts.
Model merging offers a middle ground—taking existing, high-performing specialized models and combining them. The challenge lies in identifying which models are compatible. Traditional methods rely on surface-level similarities, such as question text or human intuition, which may fail to capture the underlying cognitive patterns and structural logic that models learn during training. This paper addresses this gap by proposing a data-driven alignment strategy.
The GW-SMM Framework
The researchers propose GW-SMM to automate model compatibility assessment through a four-stage pipeline. The core innovation is the application of Gromov-Wasserstein (GW) distance to compare the internal geometry of model feature spaces.
Unlike standard distance metrics that require data to be in the same coordinate space, GW distance compares the internal relationships between points. It asks whether the relative distance between two student responses in Model A's feature space is preserved in Model B's space. This allows for the alignment of models even if their internal representations are oriented differently.
Following feature extraction and GW distance calculation, an optimal merging plan is created. Finally, the TIES-MERGING framework is used to combine the parameters of the selected models, resolving weight conflicts to create a single, unified backbone with multiple task-specific "heads."
Datasets and Tasks
The study utilized the PASTA project dataset, featuring responses from approximately 1,200 middle school students across nine physical science tasks. These tasks required students to apply scientific principles to explain phenomena, such as identifying gases based on properties like flammability and density. Responses were evaluated based on the Next Generation Science Standards (NGSS) framework, focusing on Disciplinary Core Ideas (DCIs) and Crosscutting Concepts (CCCs).
Key Results and Insights
GW-SMM was compared against three baselines: individual pre-merged models, human-knowledge-based merging, and GPT-o1-based merging. The results highlight several key findings:
- Superior Accuracy: GW-SMM achieved a Micro F1 score of 0.6872, significantly outperforming human-expert grouping (0.6741) and GPT-o1-based merging (0.6271).
- Data-Driven Alignment: The study found that GW-SMM captures nuances in student misconceptions and cognitive patterns that are not obvious from question text alone. This suggests that human intuition about science topic similarity does not always align with how a neural network perceives the underlying logic of a student's explanation.
- Efficiency Gains: By merging three models into one, the framework reduced storage requirements by 66% (a 3x reduction) while maintaining reliable scoring performance.
Conclusion and Future Work
While GW-SMM represents a significant step toward scalable AI in education, a performance gap remains between merged models and the original, unmerged individual models (which achieved a Micro F1 of 0.8291). Merging still incurs an accuracy "tax." Future research will explore adaptive layer-wise fusion, where the system learns which specific layers can be safely shared and which must remain task-specific to further minimize performance loss.
Summary updated