Forgetting Rates Add Up: Conjunctive Decay for Learning Composed Characters

When a skill is built from parts, and the parts are forgotten at different speeds, how fast is the whole skill forgotten? Most models used to schedule review treat each item or skill as having its own forgetting curve, fit more or less independently. A separate line of work shows that success on a multi-skill item behaves like a logical AND over its component skills: one missing component brings performance down to near-guessing. These two findings have never been put together. If the AND holds over time, it makes a sharp prediction. A composite item's retention is the product of its components' retentions. Under exponential decay, the composite's forgetting rate is then the sum of its components' rates. Items built from more components, or from more fragile components, should decay faster, and by an amount we can compute in advance.

We propose to test this prediction on existing math tutoring logs, and then to use it where it matters most: alphabetic-native learners acquiring Chinese characters. A character is assembled from recurring components, called radicals, and words are assembled from characters. That makes it the richest natural example of a conjunctive skill hierarchy. The same structure also offers an account of a long-standing teaching failure, the "Pinyin crutch," in which reliance on romanized pronunciation leaves learners unable to write characters. Our account treats the crutch as a shift from conjunctive to compensatory retrieval. That shift should appear as a measurable change in how the parts' forgetting rates combine, depending on whether the learner must write the character or only recognize it.

Core Claim

Much of the differences in forgetting rates across composite items can be explained by the items' structure, without fitting each item's rate separately. Each component skill $k$ has its own stability $S_k$ and its own clock. The clock resets whenever any item containing $k$ is practiced. We write the probability of answering item $i$ correctly at time $t$ as

$$p_i(t) = g_i + (1 - g_i - s_i)\prod_{k \in Q(i)} \big(m_k\, e^{-\Delta_k(t)/S_k}\big)^{w_{k,\text{task}}},$$

where the symbols are defined as follows:

  • $Q(i)$ is the set of components the item requires.
  • $m_k$ is the learner's current mastery of component $k$.
  • $\Delta_k(t)$ is the time since component $k$ was last exercised.
  • $g_i$ and $s_i$ are the item's guess and slip rates.
  • $w_{k,\text{task}} \in [0,1]$ is a learned weight saying how strictly component $k$ is required for a given kind of task (writing, recognition, or reading via pinyin).

When all weights equal one and the item is reviewed as a whole, the log of the non-guess term falls with slope $-\sum_k 1/S_k$. That is the "rates add" prediction. When the weights shrink toward zero, components can be bypassed. The product then no longer drags the item to the guess floor, and retrieval becomes effectively compensatory.

The claim has three parts, and each can be falsified:

  • Rates combine by sum. On held-out learners, composite decay rates are predicted by the sum of component rates, not their mean.
  • Shared components carry review across items. Reviewing one character slows the forgetting of an unreviewed character that shares a radical with it, and only through the shared component's factor.
  • Weights differ by task. In character learning, the weights are near one for writing and markedly lower for recognition. Pinyin-first instruction pushes the recognition weights lower still, which accelerates the decay of written form.

Relation to Prior Work

Three papers motivate the claim.

We take the phenomenon and the static gate from these papers, and add time.

Two further papers supply parts of the method.

  • DAS3H already attaches per-skill learning and forgetting counts, over expanding time windows, to items tagged with several skills (a Q-matrix). Its combination rule is additive in log-odds and therefore compensatory, which its own authors flag as a limitation. We keep its component tagging and time windows, and change only the combination rule.
  • KVFKT supplies the exponential decay with per-concept stability and the guess parameter. It treats concepts as flat and lists hierarchical dependencies as future work.

Other papers constrain the design.

  • KT² propagates evidence up and across a concept tree and handles very sparse classrooms well. Its transitions ignore elapsed time. We borrow its strength-sharing through the hierarchy but make each component's state decay in real time.
  • LefoKT warns that stacking relative-position attention on top of an explicit decay module degrades performance. We therefore do not bolt our decay onto an attention model. Instead we compare against LefoKT as a competitor.
  • Knowledge Tracing in Sequential Learning of Inflected Vocabulary is the closest precedent for tracing sub-word linguistic features with learned retention. It works on enciphered Spanish, with about 4,600 alphabetic features combined without a conjunctive structure. It also supplies our experimental template: an artificial language taught to crowdworkers to remove prior knowledge.

For the character hierarchy itself, we use existing decompositions. ZiNet provides a glyph–radical–character–word graph. Character Set Construction for Chinese Language Learning provides HanziJS two-component decompositions filtered by HSK level (the standard Chinese proficiency test). That paper also shows that radical families are often semantically opaque today. We therefore treat radicals as candidate prerequisites whose role must be confirmed by the data.

Skill-it! adds a caution for machine learners. Semantic clustering fails to recover real prerequisite structure; training-loss trajectories succeed. We will check the same thing for human learners by comparing linguistically given decompositions against ones inferred from error co-occurrence over time.

Research Program

Stage one: math logs. We test "rates add" where data already exist. On Algebra 2005, Bridge to Algebra 2006, ASSISTments 2012, and XES3G5M, we proceed in three steps. First, we estimate each component's stability $S_k$ from single-skill items only. Second, we predict the decay of multi-skill items in held-out learners, without fitting those items. Third, we compare two predictions: the sum rule of our model, and the mean rule implied by averaging or additive models. The prediction has no free parameters at the item level, so a positive result cannot come from extra capacity.

XES3G5M adds a second structural level. Its 865 knowledge components come with prerequisite routes. That lets us ask whether decay propagates down a route, meaning whether forgetting a parent accelerates errors on its descendants, or only across the components an item requires directly.

Stage two: a controlled character-learning study. We follow the enciphered-language design of the inflected-vocabulary study. Alphabetic-native crowdworkers with no Chinese learn a set of real characters paired with meanings and pinyin. We choose the characters from HanziJS/HSK decompositions so that component overlap is controlled: some characters share a radical, some share nothing, and some differ only in the phonetic component.

The study crosses four factors:

  • instruction mode: pinyin-first versus form-first;
  • test task: writing by component selection, form-to-meaning recognition, or form-to-pinyin reading;
  • retention interval: roughly one, three, and seven days;
  • sibling review: whether an unreviewed "sibling" character's radical was exercised through a different character in the meantime.

The sibling manipulation is the direct causal test of shared component clocks. The instruction-by-task interaction is the direct test of the crutch account.

Stage three: fit and compare. We fit the model with task-specific weights $w$, and compare it with the baselines below. As a final step, we let the component inventory be inferred from learners' error trajectories, following the Skill-it! logic, and ask whether the inferred inventory matches ZiNet or HanziJS radicals.

Baselines

The naive reference is a per-item exponential forgetting curve with one global stability, the uniform spaced-repetition assumption. A second naive reference fits a separate stability to every item with no structure. The second one is an upper bound on what item-level fitting can do on seen items. It cannot generalize to new characters, so it is excluded from the new-item test. All models share the same train/test learner splits and the same hyperparameter search budget. The neural models also get the same compute.

The strongest structured competitor is DAS3H, given exactly our component Q-matrix and time windows. On Algebra05 it reports an AUC of 0.826, against 0.775 for DASH and 0.771 for IRT. We treat 0.826 as the bar to match on that dataset. We also report DAS3H on the other datasets under our splits, because the file gives no reported DAS3H number for Bridge06 or ASSISTments 2012–13.

The strongest time-aware neural competitors are the following:

  • KVFKT, which reports an AUC of 0.834 on ASSISTments 2012.
  • LefoKT, which reports roughly a 0.4% AUC gain over AKT on BD2006 and stable AUC when evaluated on sequences longer than training.
  • IEKT, which XES3G5M identifies as its top performer. The file gives no reported value, so there is no reported result found on Lacuna for IEKT's AUC on XES3G5M.

Each neural model is given the component identities as extra embeddings, so that it has access to the same structure. For the character study, no reported result found on Lacuna exists for any model. The closest reference is the PKT result of 0.449 predictive accuracy against 0.429 for an LSTM, on its enciphered-Spanish data. That result is not directly comparable to ours.

The key ablation swaps the product of retentions for DAS3H's additive log-odds combination. It keeps identical component features, stabilities, and clocks. This isolates the conjunctive rule. A second ablation removes shared clocks, so that each component's clock resets only when that very item is reviewed. This isolates cross-item transfer through components.

What counts as success. We do not require a large gain in pooled AUC, because pooled AUC is dominated by single-skill items. The model beats the baselines if it does all of the following:

  • matches DAS3H and the neural models on pooled AUC within noise;
  • clearly beats them on multi-skill and new items at long retention intervals;
  • beats the additive ablation on the sum-versus-mean test.

Why not just…?

Why not just use DAS3H with radicals tagged as skills? This is the most serious alternative. It already has component-specific forgetting and handles new characters through their components. The two models differ only in the combination rule. The additive ablation is literally this alternative, so the experiment settles the question directly. If additive log-odds predicts multi-skill decay as well as the product does, the conjunctive claim fails and we will report that. The prior evidence against it is static: The Composition Effect found that a flexible learned composition collapses to an AND gate. That finding does not guarantee the AND survives over time, which is exactly what we test.

Why not just give a time-aware attention model (LefoKT, AKT, IEKT) the component embeddings and let it learn the interaction? With enough data it might. KT² shows that such models fail in sparse settings where hierarchical models hold up. A character-learning study with a few hundred learners is sparse. The more important point is that a neural model's fit does not produce a testable quantity such as the sum rule or the writing-versus-recognition weights. We compare accuracy on equal footing, and report whether the neural models' predicted decay for multi-skill items, extracted by probing at controlled delays, happens to follow the sum rule.

Why not just fit a separate forgetting rate to each character, as spaced-repetition systems do? On characters the learner has already seen, this may predict well. It cannot say anything about a new character, or about how reviewing one character affects its siblings. The new-item and sibling-review tests are exactly where it has nothing to offer. Modelling Student Learning and Forgetting already shows that a single shared decay rate is wrong. We include the per-item model anyway, so that we know how much predictive accuracy structure buys on seen items.

Decisive Checks

The first check is the zero-parameter sum-rule test on the math logs. Multi-skill decay slopes should be predicted by the summed component rates. The sum should explain more variance than the mean does, and should beat the additive ablation. If the mean wins, the core claim is wrong.

The second check is the sibling-review manipulation. Exercising a radical through character A should reduce errors on unreviewed character B, and the reduction should appear in writing more than in recognition. A null result would refute shared component clocks.

The third check is the crutch interaction. Pinyin-first learners should show lower fitted recognition weights and faster decay of written form than form-first learners. Their recognition decay should be similar. If the two groups' weights do not differ, the Pinyin crutch is not a change in how retrieval combines components, and some other account is needed.

Finally, we compare radicals given by ZiNet and HanziJS with components inferred from error trajectories. The comparison tells us whether linguistic decompositions are the actual units that learners forget.