Introduction
In AI-driven education (AIEd), accurately assessing a student's knowledge state is critical for effective personalization. Traditional Knowledge Tracing (KT) models typically employ a dichotomous approach, focusing solely on whether a student's response is correct or incorrect. This binary classification overlooks significant diagnostic information contained within the specific incorrect options (distractors) a student selects. Conversely, Option Tracing (OT) predicts the specific choice made by a student but often fails to leverage the explicit correctness signal, potentially tracking behavioral patterns rather than actual knowledge.
To address these limitations, researchers have developed DP-MTL (Dichotomous-Polytomous Multi-Task Learning), a framework that integrates both KT and OT objectives to achieve a more granular and accurate assessment of student learning.
Methodology: The DP-MTL Framework
The core of DP-MTL is a multi-task objective function that balances dichotomous correctness prediction with polytomous option choice prediction. The combined loss function is defined as:
$$L_{DP} = \lambda L_D + (1 - \lambda) L_P$$
where $L_D$ represents the standard KT loss and $L_P$ represents the OT loss. The hyperparameter $\lambda \in [0, 1]$ allows the model to balance these two objectives.
KT as a Regularization Mechanism
A key theoretical contribution of this work is the demonstration that the KT objective acts as a regularizer for the OT task. In instances where a student provides an incorrect response, the KT loss encourages the model to reduce the probability of the correct option while distributing the remaining probability across all distractors. This prevents the model from over-fitting to a single incorrect choice and leads to more stable student representations.
Position-Agnostic Architecture
To avoid "positional bias"—where models learn a student's tendency to pick a specific letter (e.g., 'C')—DP-MTL utilizes option embeddings. This architecture allows the model to remain agnostic to the number and order of choices, focusing instead on the semantic content of the options. Furthermore, the framework is model-agnostic and can be integrated into various architectures such as Item Response Theory (IRT), Neural Matrix Factorization (NMF), and Bidirectional LSTMs (BiDKT).
Experimental Evaluation
The framework was evaluated on two datasets: ENEM (a dense dataset from Brazil's national college entrance exam) and TOEIC (a sparse dataset from an online tutoring system).
Key Results
The integration of dichotomous and polytomous tasks yielded significant improvements across multiple metrics:
- Score Prediction (SP): DP-MTL significantly outperformed baseline models in predicting final exam scores. On the TOEIC dataset, the Mean Absolute Error (MAE) was reduced by up to 24.9% compared to standard KT models.
- Knowledge Tracing Performance: Even when the primary goal is binary correctness prediction (AUC), the inclusion of the option-tracing task provided a performance boost, particularly in sparse data environments.
- Optimal Task Balancing: Empirical results consistently showed that the best performance is achieved when $\lambda$ is between 0 and 1, typically in the 0.5 to 0.7 range. This confirms that the two tasks are complementary and that multi-task learning is superior to focusing on either task in isolation.
Conclusion
DP-MTL demonstrates that "no task should be left behind" in student assessment. By simultaneously learning from correctness and specific option choices, the model builds a holistic representation of student knowledge. This granular understanding not only improves performance on standard benchmarks but also provides a foundation for more effective personalized content recommendation and diagnostic feedback in digital learning environments.
Summary updated