Adaptive learning systems decide, student by student, what to teach next. A knowledge-tracing or cognitive-diagnosis model estimates each student's latent proficiency from their answer history, and a selection policy uses that estimate to pick the next question or practice set. These systems are now being rebuilt around deep measurement models and reinforcement-learning selection policies (Survey of Computerized Adaptive Testing: a Machine Learning Perspective), often drawing items from third-party question banks and training data from platform logs. This note proposes that this pipeline has an unexamined integrity failure mode: a backdoor attack in which a small fraction of assessment items carries a subtle trigger — a recurring phrasing pattern, distractor template, or stylized wording — such that students coached on the trigger (or a measurement model trained on trigger-correlated responses) appear to master skills they do not. The victim is not a classifier's label but a latent proficiency estimate, and the harm lands exactly where prior work showed the stakes are highest: The Impact on Individualizing Student Models on Necessary Practice Opportunities found that a mere population-level approximation of student models already pushes 17% of students past a mastery threshold while their true individual mastery is 60% or below. A deliberate trigger converts that miscalibration error rate into an attack-controlled dial.
What makes this problem hard, and what makes learner heterogeneity central rather than incidental, is the defense side. The natural defenses transferred from the backdoor literature are all blind to individual learning dynamics. Entropy-based coreset filtering (Anti-Backdoor Coreset Selection via Cumulative Entropy) treats rapidly-learned, low-uncertainty patterns as suspicious shortcuts — but in education, fast learners genuinely look like that. Run-time input perturbation (STRIP: A Defence Against Trojan Attacks on Deep Neural Networks) blends inputs to test for trigger dominance, but blending or perturbing a psychometric item destroys its validity, and STRIP already struggles with partial, source-specific backdoors — which is precisely what a trigger aimed at one coached subpopulation is. Prompt-level defenses (ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attacks) assume the attack lives in a demonstration context, not in a measurement model's training data and item bank. So the detector we actually need must separate two things that look identical in aggregate: a genuinely fast learner, and a trigger-assisted student.
Core Claim
The separating signal is temporal, not distributional. A genuinely fast learner's advantage grows along a plausible individual trajectory — it appears gradually, precedes their correct answers, and transfers to untriggered items measuring the same skill. A trigger-assisted student's performance spikes only on the specific items that carry the trigger, appears without any preceding growth in ability, and does not transfer. By fitting a dynamic model in the spirit of D-BIRD, which decomposes ability into a shared cohort trend $\mu_t$ plus an individual deviation $\beta_{i,t}$ that evolves as a random walk, we can test each candidate item for whether its responses are better explained by a plausible $\beta_{i,t}$ trajectory or by an item-level effect that violates learning dynamics. Heterogeneity, in other words, is the detector: the model of how individuals legitimately deviate from the cohort is exactly what distinguishes a fast learner from a poisoned item. No prior work in the file does this — the backdoor literature works on images and standard NLP benchmarks, and the psychometric literature has no adversarial analysis at all.
The Attack
We define and evaluate two threat models on the adaptive-testing stack. In the bank-poisoning (test-prep) attack, an item vendor injects a recurring template into a small fraction of bank items, and a coaching program teaches students to recognize it — a clean-label attack, since every individual response is a genuine, human-produced answer. In the data-poisoning attack, the training corpus for a deep measurement model such as NeuralCD contains trigger-carrying items whose responses correlate with correctness, so the learned diagnostic function maps trigger features to mastery signals; at deployment, any student who encounters a triggered item gets an inflated proficiency estimate. In both cases the target metric is not classification accuracy but downstream instructional decisions: false mastery (a student crossing the 95% mastery threshold while their trigger-free true mastery is at or below 60%, the exact failure definition used by The Impact on Individualizing Student Models on Necessary Practice Opportunities), and corruption of the item-selection policy. This is structurally different from the closest existing attacks: Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning poisons demonstration prompts to flip a classifier's output, whereas here the poisoned object is a latent trait estimate that feeds an adaptive control loop, and the attacker's leverage is amplified by the heterogeneity the system is built to serve. The persona-jailbreak line (Disentangling Intent from Role: Adversarial Self-Play for Persona-Invariant Safety Alignment) shows that conditioning on a subpopulation can shift model behavior, but it targets safety refusal, not measurement integrity.
The Defense: Trajectory-Consistency Detection
We fit, per skill, a dynamic item-response model in which the probability that student $i$ answers item $j$ correctly at time $t$ is
$$P(r_{ijt}=1)\;=\;\sigma\!\left(\mu_t + \beta_{i,t} + b_j + \gamma_j\, z_j\right),$$
where $\mu_t$ is the cohort trend, $\beta_{i,t}$ the student's individual deviation following a random walk as in D-BIRD, $b_j$ the item difficulty, $z_j$ an indicator for a candidate trigger cluster (found by clustering item templates), and $\gamma_j$ the trigger effect. Following D-BIRD's Pólya-Gamma augmentation, the model is fit by Gibbs sampling. An item is flagged when two conditions hold jointly: the posterior strongly favors $\gamma_j \neq 0$, and correctness on that item cannot be attributed to the student's $\beta_{i,t}$ path — correct answers arrive before any deviation growth that would justify them, and they do not predict performance on temporally adjacent untriggered items of the same skill. This "no-growth, no-transfer" test is the key ingredient; a static item-misfit statistic without the trajectory component is our central ablation.
Baselines
The do-nothing reference is the poisoned pipeline with no defense: a NeuralCD measurement model with an RL selection policy (NCAT) on Eedi2020, run on the poisoned bank, which measures how much false-mastery inflation the attack achieves unimpeded; its clean-data reference point is the NCAT AUC@20 of 79.12% versus 74.99% for random selection reported by Survey of Computerized Adaptive Testing: a Machine Learning Perspective, and its harm anchor is the 17% false-mastery rate that population-level miscalibration already produces on ASSISTments data in The Impact on Individualizing Student Models on Necessary Practice Opportunities. The strongest existing competitor is ABCS, cumulative-entropy coreset selection from Anti-Backdoor Coreset Selection via Cumulative Entropy, which is the best-performing data-level defense in the file — it reports a Defense Effectiveness Rate above 95% on CIFAR10 with attack success rate reduced to roughly 1%, and 97% on Tiny-ImageNet, with no clean-accuracy drop where prior defenses lose 5–10%. We will port its cumulative-entropy filtering to response logs, tune it on the same held-out validation split, under the same compute and search budget as our detector, and evaluate it on both attack neutralization and a second axis it has never been tested on: the rate at which it excludes genuinely fast learners. The second defense baseline is ICLShield from ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attacks, adapted by prepending high-similarity, high-confidence clean response histories; it reports an average attack-success-rate reduction of 29.14% across NLP benchmarks (46.21% on SST-2), which sets the bar our detector must clear by a wide margin, since a 29-point reduction against a 95%-success attack leaves a large residual. For the data-poisoning threat model we add Fine-Pruning (Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks) as the weight-level baseline. The decisive ablation replaces our full detector with a static, population-level item-misfit statistic (the same model without the $\beta_{i,t}$ trajectories), which isolates whether learning dynamics, rather than mere outlier detection, do the work. Beating the field means: false-mastery rate on poisoned data driven back to within two points of the clean baseline, NCAT's clean AUC@20 held within one point of the reported 79.12%, and a fast-learner exclusion rate far below every defense baseline.
Why not just…?
Why not just filter the data with ABCS-style entropy, as-is? Because its core heuristic is inverted in this domain. ABCS assumes that samples learned rapidly with early confidence are adversarial shortcuts; D-BIRD shows that individual deviations from the cohort trend are real, persistent, and legitimate — gifted or well-prepared students master concepts almost immediately. Our heterogeneity-fairness experiment directly measures this: we label the top quantile of $\beta$-growth students on clean data and record each defense's exclusion rate against them. If ABCS excludes them at a high rate while our detector does not, the simpler method is disqualified for this setting regardless of its attack numbers; the ABCS authors themselves note it fails against adaptive attacks like A-Blend, so it enters the comparison with a known weakness on top of the predicted fairness failure.
Why not just prepend clean examples, ICLShield-style? Three reasons, two of which we will demonstrate empirically. First, in adaptive testing the attack is not in a prompt's demonstration set but in the item bank and the measurement model's training corpus, so prepending clean context never touches the poisoned object; we run the adapted version anyway, and expect it to underperform its reported 29.14% average reduction. Second, its selection heuristics are pedagogically corrosive: choosing high-confidence, high-similarity exemplars biases instruction toward standard material, which is precisely what struggling learners — the population individualized instruction exists to serve — do not need. Third, even at its reported strength it is partial; on GSM8K under BadChain it left attack success at 39.58% after defense, which in a mastery-gating context is still a catastrophic false-advancement rate.
Why not just abandon neural measurement models and use classic IRT, which has audited item parameters? This does not dodge the attack, because the attack does not need the neural model. Trigger-correlated responses poison the estimation data, so even a classical item's fitted difficulty and discrimination are corrupted, and heterogeneous coaching subpopulations distort the fit in ways population-level item statistics cannot localize. Meanwhile, retreating costs the documented personalization gains: the data-driven selection policies reviewed in Survey of Computerized Adaptive Testing outperform heuristics (AUC@20 of 79.12% versus 74.99% for random selection on Eedi2020). Our defense is deliberately model-agnostic in spirit — the trajectory test works on response logs regardless of whether the deployed measurement model is NeuralCD or a Rasch model — so adopting it does not force a choice between auditability and the deep stack.
Proposed Experiments
The primary testbed is Eedi2020 with a NeuralCD measurement model and an NCAT-style selection policy, with trigger items injected at 0.5%, 1%, and 5% of the bank under both threat models. Headline metrics: attack success (share of coached or affected students crossing the mastery threshold early), false-mastery rate under the ≤60%-true-mastery definition, and NCAT's AUC@20 degradation versus the clean 79.12% reference. The second testbed is the ASSISTments corpus (42 skills, 265 students) with Bayesian knowledge tracing, where the harm metric is the distortion of the Expected Number of Practice Opportunities; the clean-data anchors are the 55% over-practice ("boredom tax") and 17% false-mastery figures from The Impact on Individualizing Student Models on Necessary Practice Opportunities, and the attack's signature would be a directed version of that error — over-advancement for affected students only. No reported result exists on Lacuna for these corpora under any backdoor attack, which is itself the gap. Scale-up replication on Junyi, EdNet, and MOOCCube (for which no reported numbers on any related metric were found on Lacuna) tests whether the detector survives sparser logs. Three decisive checks close the loop: the heterogeneity-fairness comparison (exclusion rates on true fast learners, where we predict ABCS fails and our detector does not); an adaptive attacker who spreads the trigger across many templates and entangles it with a difficult skill to mimic high-entropy benign items, probing both our detector's clustering and ABCS's known A-Blend weakness; and the trajectory ablation, which must show the static misfit version collapsing toward population-level detection — flagging legitimate subpopulations along with the attack — while the full model holds precision. If the dynamic component adds nothing, the core claim is wrong and the honest conclusion is that backdoored items in adaptive assessment are just ordinary item-misfit outliers; that outcome would itself be the first adversarial characterization of measurement-model poisoning in this literature.
References
- Survey of Computerized Adaptive Testing: A Machine Learning Perspective
- The Impact on Individualizing Student Models on Necessary Practice Opportunities
- Anti-Backdoor Coreset Selection via Cumulative Entropy
- STRIP: a defence against trojan attacks on deep neural networks
- ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attacks
- Dynamic Bayesian Item Response Model with Decomposition (D-BIRD): Modeling Cohort and Individual Learning Over Time
- Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning
- Disentangling Intent from Role: Adversarial Self-Play for Persona-Invariant Safety Alignment
- Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks