TL;DR
Standard learner models often mistake the environmental barriers faced by neurodivergent (ND) students for a lack of innate ability. This paper introduces IRT-ZILM, a zero-inflated model that decouples student ability from "contextual inequity," enabling fairer assessment and personalized learning adjustments that can significantly improve outcomes for students with conditions like dyslexia and dyscalculia.
What Problem Does This Paper Address?
In the UK alone, an estimated 15% of the population is neurodivergent. Each neurodivergent condition (NDC)—such as dyslexia, dyscalculia, or Sensory Processing Disorder (SPD)—uniquely affects how a student processes information.
The core challenge in Educational Data Mining (EDM) is confounding context. A student with dyslexia might fail a math problem not because they lack mathematical ability, but because the question is presented as a dense block of text. Traditional Learner Models (LMs), like Item Response Theory (IRT), see an incorrect answer and simply lower the student's ability estimate. This leads to a "legacy of potentially poor learning contexts," where LMs become biased, underestimating ND students and offering inequitable recommendations.
The authors identify three major gaps:
- ND ability is inaccurately modeled due to LM shortcomings.
- Learning environments rarely consider the most suitable context for a student to express their ability.
- There is a severe lack of datasets and simulation tools for assessing equity in ND contexts.
Methods
The authors propose the IRT-based Zero-Inflated Learner Model (IRT-ZILM).
Zero-Inflated Models (ZIMs) are typically used for count data with an excess of zeros. The authors adapt this logic to binary student performance (correct/incorrect). In their formulation, an incorrect answer (a "zero") has two explicit explanations:
- Low Ability ($p$): The student lacks the knowledge to solve the item.
- Low Learning Quality Factor ($\pi$): The item's Delivery and Response Type (DRT) is unsuitable for the student's specific NDC, effectively "sabotaging" their performance regardless of ability.
The model is defined as: $$Pr(Y = y) = \begin{cases} \pi + (1 - \pi) \cdot (1 - p) & \text{if } y = 0 \\ (1 - \pi) \cdot p & \text{if } y = 1 \end{cases}$$
Here, $p$ is derived from standard IRT (ability vs. difficulty), while $\pi$ is parameterized by the student's NDC and the item's DRT features. This allows the model to "blame" the context rather than the student's intelligence for certain failures.
Datasets
Because real-world datasets labeled with specific NDCs and varied delivery formats are virtually non-existent, the authors developed a comprehensive simulation environment.
They generated a synthetic dataset where ND and neurotypical (NT) students share the same underlying ability distribution. The simulation includes:
- Student Traits: Ability, and the presence of Dyslexia, Dyscalculia, or SPD.
- Item Traits: Difficulty, subject (Math/English), information density, and specific Delivery and Response Types (DRT) (e.g., Read vs. Listen, Written vs. Speak).
This ground-truth data allows the authors to measure exactly how much an estimation model deviates from a student's "true" unobserved ability.
Key Insights and Ideas
- LQF as a Measure of Inequity: The authors define the Learning Quality Factor (LQF) as $1-\pi$. As shown in Figure 1, a low LQF acts as an upper asymptote on success probability. Even a high-ability student cannot succeed if the delivery format is inaccessible.
- Identifying NDCs through Interaction: By comparing performance across different subjects and DRTs, the model can flag potential NDCs. For example, a student who performs well on "Listen" items but fails "Read" items shows a pattern characteristic of dyslexia.
- Decoupling Confounders: By separating $\pi$ and $p$, IRT-ZILM provides an interpretable framework. Educators can see why a student is struggling—is it a knowledge gap or a delivery barrier?
Key Results
- Superior Parameter Recovery: IRT-ZILM is significantly better at estimating true student ability. As shown in Figure 4, standard IRT is heavily biased, consistently underestimating students with 1 or 2 NDCs. IRT-ZILM estimates remain reliable and centered on the diagonal (perfect estimation).
- Quantitative Accuracy: In recovering student ability, IRT-ZILM achieved a Pearson correlation of 0.993, compared to 0.839 for standard IRT and 0.955 for Knowledge Tracing Machines (KTM).
- Improving Learning Opportunities: The authors demonstrated that by "actively selecting" the most suitable DRT for a student, they could achieve a "lift" in successful learning attempts of 1.43x for students with one NDC and 1.89x for those with two NDCs (Table 4).
Limitations and Future Work
- Synthetic Data Reliance: The primary limitation is the use of simulated data. While necessary due to data scarcity, the weight vectors used to simulate NDC effects are subjective.
- Identification vs. Diagnosis: While the model can identify students who struggle in specific contexts, the authors note that formal statistical hypothesis testing is needed to distinguish between a student needing support and a formal NDC diagnosis.
- Future Directions: The authors plan to expand the model to new domains and explore more complex educational contexts beyond simple question-answering. They also suggest that this framework could be used by teachers to audit their own content for accessibility.
Summary updated