Symbolic Cognitive Diagnosis via Hybrid Optimization for Intelligent Education Systems

Junhao Shen, Hong Qian, Wei Zhang, Aimin Zhou

Abstract

Cognitive diagnosis assessment is a fundamental and crucial task for student learning. It models the student-exercise interaction, and discovers the students' proficiency levels on each knowledge attribute. In real-world intelligent education systems, generalization and interpretability of cognitive diagnosis methods are of equal importance. However, most existing methods can hardly make the best of both worlds due to the complicated student-exercise interaction. To this end, this paper proposes a symbolic cognitive diagnosis (SCD) framework to simultaneously enhance generalization and interpretability. The SCD framework incorporates the symbolic tree to explicably represent the complicated student-exercise interaction function, and utilizes gradient-based optimization methods to effectively learn the student and exercise parameters. Meanwhile, the accompanying challenge is that we need to tunnel the discrete symbolic representation and continuous parameter optimization. To address this challenge, we propose to hybridly optimize the representation and parameters in an alternating manner. To fulfill SCD, it alternately learns the symbolic tree by derivative-free genetic programming and learns the student and exercise parameters via gradient-based Adam. The extensive experimental results on various real-world datasets show the superiority of SCD on both generalization and interpretability.

Introduction

In Intelligent Education Systems (IES), understanding the specific reasons behind a student's performance is critical for personalized learning. This task, known as Cognitive Diagnosis Assessment (CDA), maps student-exercise interactions to proficiency levels across knowledge attributes. Historically, researchers have faced a trade-off: traditional models like IRT or DINA are highly interpretable but often too simple to capture complex learning patterns, while neural models like NCDM offer high accuracy but function as "black boxes."

To bridge this gap, the Symbolic Cognitive Diagnosis (SCD) framework is proposed. It aims to achieve both high generalization (accuracy) and transparency by representing student-exercise interactions through symbolic trees, which are human-readable mathematical expressions.

The SCD Framework

The core innovation of SCD lies in its Hybrid Optimization approach. The framework treats cognitive diagnosis as a dual-optimization problem involving both discrete symbolic structures and continuous parameters.

Overview of the Symbolic Cognitive Diagnosis (SCD) framework, illustrating the flow from diagnostic factors through the hybrid optimization loop to the final symbolic tree implementation.

As shown in the framework overview, the process involves two main components:

  1. The Symbolic Tree (Discrete Optimization): Instead of a fixed function or a neural network, the interaction function is represented as a mathematical expression tree. Using Genetic Programming (GP), the model evolves the best functional form from a set of operators (e.g., $+$, $\times$, $sigmoid$, $tanh$).
  2. Parameter Optimization (Continuous Optimization): While the tree defines the structure, specific values for student proficiency ($P$) and exercise features (difficulty, discrimination) are optimized using the Adam gradient-based optimizer.

To bridge these two domains, SCD employs an alternating training strategy: it fixes the tree structure to optimize parameters, then fixes the parameters to evolve the tree. Furthermore, the framework enforces a Monotonicity Assumption—ensuring that a student's probability of a correct answer increases with their proficiency—by restricting the GP to monotonic mathematical operators.

Experimental Evaluation

The SCD framework was evaluated against state-of-the-art models using four real-world datasets: Math1, Math2, FracSub, and NeurIPS2020.

Key findings include:

  • Competitive Accuracy: The SCD implementation (SCDM) performed on par with or better than leading neural models (such as KaNCD) across metrics like AUC, Accuracy, and F1-score.
  • Superior Interpretability: SCDM consistently achieved the highest Degree of Agreement (DOA) scores. This indicates that the proficiency levels it assigns are more consistent with actual student performance patterns than those of black-box models.
  • Transparent Logic: Case studies revealed that the learned symbolic trees clearly separate different pedagogical factors, such as distinguishing between the impact of student proficiency and exercise discrimination.

Conclusion and Future Work

SCD demonstrates that symbolic regression can effectively capture the latent logic of human cognition while maintaining the transparency required for educational trust. By combining evolutionary algorithms with gradient-based optimization, the framework provides a robust solution to the generalization-interpretability dilemma. Future research will focus on improving the theoretical convergence of the hybrid optimization process and managing the search space complexity as the number of knowledge attributes increases.

Summary updated

Abstract

Cognitive diagnosis assessment is a fundamental and crucial task for student learning. It models the student-exercise interaction, and discovers the students' proficiency levels on each knowledge attribute. In real-world intelligent education systems, generalization and interpretability of cognitive diagnosis methods are of equal importance. However, most existing methods can hardly make the best of both worlds due to the complicated student-exercise interaction. To this end, this paper proposes a symbolic cognitive diagnosis~(SCD) framework to simultaneously enhance generalization and interpretability. The SCD framework incorporates the symbolic tree to explicably represent the complicated student-exercise interaction function, and utilizes gradient-based optimization methods to effectively learn the student and exercise parameters. Meanwhile, the accompanying challenge is that we need to tunnel the discrete symbolic representation and continuous parameter optimization. To address this challenge, we propose to hybridly optimize the representation and parameters in an alternating manner. To fulfill SCD, it alternately learns the symbolic tree by derivative-free genetic programming and learns the student and exercise parameters via gradient-based Adam. The extensive experimental results on various real-world datasets show the superiority of SCD on both generalization and interpretability. The ablation study verifies the efficacy of each ingredient in SCD, and the case study explicitly showcases how the interpretable ability of SCD works.
BibTeX
@inproceedings{shen2024symbolic,
  author = {Junhao Shen and Hong Qian and Wei Zhang and Aimin Zhou},
  title = {{Symbolic Cognitive Diagnosis via Hybrid Optimization for Intelligent Education Systems}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i13.29413},
}
Figure