When a learner of Arabic writes a word wrong, one of two things usually happened. The writer knew the right form and their fingers slipped, or the writer does not yet know the rule, for example which gender a verb must agree with or where a clitic attaches. The written evidence can look the same in both cases. The patterns differ, though. Slips are scattered and inconsistent, while knowledge gaps recur whenever the same grammatical feature comes up. Current Arabic correction systems ignore this difference. They pick corrections using fixed edit costs, or error distributions learned from native writers, and they fail badly on learner text. In the second QALB shared task, the top system reached $F_1 = 72.87$ on native text but only $F_1 = 41.20$ on learner essays (The Second QALB Shared Task on Automatic Text Correction for Arabic).
This note proposes a model that learns both kinds of error from learner text that nobody has corrected. It borrows the "slip and guess" latent-mastery models from educational data mining, which infer whether a student has mastered a skill from noisy test answers. It applies that idea to morphological edit paths. An edit path is the chain of spelling changes and grammatical-feature changes that turns the intended word into the written one. The parameters are estimated by expectation-maximization (EM), an algorithm that alternates between guessing the hidden quantities under the current parameters and re-fitting the parameters to those guesses. Here the hidden quantities are each token's intended word and each writer's mastery of each feature.
Core Claim
A learner's errors carry a signal that no current Arabic correction system uses: whether a feature error repeats within the same writer. Treat per-writer, per-feature mastery as a hidden variable, and treat the joint orthographic-morphological edit path as the observation. EM can then separate mechanical slips from competence errors without parallel corrections. The resulting error model, adapted to the target learner population, should improve correction on learner text when used to rerank the candidates of a strong neural corrector. It should also yield a writer-level diagnosis that predicts which errors that writer will make later.
The novelty is not "learn edit costs with EM." Stochastic edit weights estimated from data are an old idea in spelling correction. That is also why we do not claim it. The new part has two pieces. The hidden variable lives in morphological feature space, as ARETA's ortho-morph paths define it. And a writer-level mastery variable makes feature errors dependent within a writer while leaving slips independent.
Where This Sits
Three strands of prior work motivate the gap.
ARETA (Automatic Error Type Annotation for Arabic) computes the shortest joint orthographic-morphological path between an erroneous word and its correction using CAMeL Tools. It reaches 85.8% micro-F1 against human error-type labels on the Arabic Learner Corpus. However, ARETA needs the gold correction and uses fixed costs. We reuse its path space as our latent space, but infer paths when the correction is unknown.
SWEET (Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Study) turns weighted-Levenshtein alignments into subword edit tags and prunes rare tags. AraBART with morphological features and 13-class error-detection tags (Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigation) reaches $F_{0.5} = 79.6$ on QALB-2014 and $82.7$ on ZAEBUC. Both are supervised on parallel data dominated by native writers. We do not replace them; we supply a target-adapted error prior on top of them.
The PELIC analysis (Identifying and analyzing 'noisy' spelling errors in a second language corpus) models aggregate error counts with hurdle models. It finds that L1 effects appear at low proficiency and fade at higher levels. It also states that it cannot separate real-word "clang associates" or systematic interference from slips. This is exactly the separation we target.
The estimation machinery comes from education. Brunskill (Estimating Prerequisite Structure From Noisy Data) uses EM over hidden binary mastery with slip and guess noise. She fixed those rates by hand at $p_s = 0.3$ and $p_g = 0.2$ and named this as the main limitation. COMMAND (Joint Discovery of Skill Prerequisite Graphs and Student Models) extends this to structural EM over skill graphs and reports AUC 0.803 and 0.775 on math tutoring data.
Their experience also constrains our design. Freeing noise parameters per item creates non-convexity and identifiability trouble. We therefore tie slip rates by edit class rather than per word. We also initialize from counts on native parallel data, not from scratch. The Columbia QALB-2014 pipeline (The Columbia System in the QALB-2014 Shared Task on Arabic Error Correction) combined MADAMIRA with single-pass maximum-likelihood mappings. It shows that supervised counting already works in-domain; our claim concerns adaptation where corrections are absent.
Model
For a writer $w$, each observed token $x$ is generated in four steps:
- A context model proposes an intended morphological analysis $a$ (lemma plus features).
- The writer produces a believed analysis $a'$. For each feature $f$ present in $a$, the writer changes it with probability $\epsilon_f$ if they have mastered $f$ ($m_{w,f} = 1$), or with probability $\gamma_f$ if they have not ($m_{w,f} = 0$).
- The believed analysis $a'$ is realized as a surface string $s$ through the morphological generator.
- A slip channel with class-tied character-edit probabilities $\theta$ turns $s$ into the observed $x$.
The likelihood of writer $w$'s text is
$$ p(x_{1:n}\mid w) = \sum_{m_w} \prod_f \pi_f^{m_{w,f}}(1-\pi_f)^{1-m_{w,f}} \prod_{i=1}^{n} \sum_{a_i, a'_i} p_{\text{LM}}(a_i\mid \text{context})\, p(a'_i \mid a_i, m_w)\, p_\theta(x_i \mid s(a'_i)). $$
Here $\pi_f$ is the population rate of mastering feature $f$.
The inner sums run over a pruned candidate set. That set holds analyses from CAMeL Tools within a bounded edit distance of $x_i$, plus the top-$k$ candidates from a neural corrector. This keeps the E-step tractable despite the combinatorial blow-up that morphologically rich languages cause (Interactive Word Completion for Morphologically Complex Languages).
The model can tell slips from gaps because the mastery variable is shared across all of a writer's tokens that carry feature $f$. A consistent gender-agreement change across an essay pushes the posterior toward $m_{w,f} = 0$. A single stray letter substitution is absorbed by $\theta$.
In the M-step, $\theta$, $\epsilon_f$, $\gamma_f$ and $\pi_f$ get closed-form updates from expected counts. We add Dirichlet priors centered on the counts obtained by running ARETA over the gold paths of native QALB-2014 training data. We fix the label-switching ambiguity by requiring $\gamma_f > \epsilon_f$.
Research Program
The work has three parts.
First, we build the candidate generator and the edit-path lattice over CAMeL Tools analyses. We verify that, when gold corrections are supplied, its Viterbi paths agree with ARETA's error types on the Arabic Learner Corpus. This guards against a broken latent space.
Second, we test identifiability on synthetic learners. We take corrected QALB text, split it into pseudo-essays, and assign each pseudo-writer a known mastery profile and a known slip rate. We inject errors accordingly and measure how well EM recovers $m_{w,f}$ and the rates as essay length and number of writers vary. This tells us the minimum text per writer at which the within-writer recurrence signal becomes usable.
Third, we run the model on real learner text. We run EM on the source side only of QALB-2015 L2 training essays, discarding their corrections. The adapted channel is then used to rerank the candidates of SWEET and of AraBART+Morph on the L2 test set. We also read off per-essay mastery posteriors for the diagnostic experiment.
Baselines
The naive reference point is the do-nothing system, which copies the source unchanged. Its M2 score fixes the floor for learner text.
Above it sits ARETA-style correction. This uses the same candidate generator and context model, but with fixed uniform edit costs and no mastery variable. It isolates the value of estimating anything at all.
The supervised-counting baseline estimates the full error channel by counting gold ARETA paths on native QALB-2014 training data, then applies it to learner text without adaptation. This is the Columbia-style single-pass estimator (The Columbia System in the QALB-2014 Shared Task on Arabic Error Correction) recast in our path space. If EM on unlabeled learner text cannot beat it, the adaptation claim fails.
The strongest existing methods are AraBART+Morph+GED (Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigation) and SWEET (Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Study). We retrain both on QALB-2014 plus the QALB-2015 L2 training set with gold corrections, so they see more supervision than our channel does. The reported reference numbers are $F_{0.5} = 79.6$ on QALB-2014 for AraBART+Morph and $F_1 = 41.20$ for the best QALB-2015 L2 shared-task system (UMMU-1). No reported L2-track result for SWEET or AraBART+Morph was found on Lacuna, so we will run them ourselves under identical tokenization, the same M2 scorer, and a matched hyperparameter search budget.
Two ablations isolate the key ingredient. The first removes the mastery variable so all writers share one confusion distribution; this is ordinary EM noisy-channel adaptation. The second keeps writer mastery but drops the morphological layer, so errors are modeled over characters only.
Beating the strongest method means two things. The reranked system must improve L2 test $F_{0.5}$ and $F_1$ over the same neural corrector without reranking, with significance under paired bootstrap resampling over essays. And the full model must beat the no-mastery ablation, since otherwise writer mastery adds nothing beyond generic adaptation.
Why not just…?
Why not just fine-tune AraBART or SWEET on the learner data? This is the main threat. The QALB-2015 L2 training set is small, about 43,000 words, but a pretrained model may still absorb learner-specific errors from it. We therefore include exactly this system as a baseline with gold L2 corrections. Our channel sees none of those corrections.
If fine-tuning closes the gap entirely, the correction claim is dead. The diagnostic claim would then be the remaining contribution, since a fine-tuned corrector outputs corrections, not a per-writer account of which rules are unknown. The regime we expect to matter is learner populations with no parallel data. We test it by varying how much L2 gold the neural baseline gets, from 0% to 100%, and finding the crossover point.
Why not just count error paths from parallel data? Where gold corrections exist in-domain, counting is simpler than EM and nearly as good, and we concede this. The QALB-2015 gap between native and learner performance (The Second QALB Shared Task on Automatic Text Correction for Arabic) indicates that native-estimated counts transfer poorly to learners. The supervised-counting baseline measures exactly how poorly, and EM adaptation must recover part of that loss to justify itself.
Why not just self-train, pseudo-labeling learner text with the best corrector and re-estimating? Self-training is essentially hard-EM without the mastery variable, so it is a genuine competitor for the adaptation gain. We run it with the same unlabeled L2 text and the same number of rounds. We expect it to match our no-mastery ablation. Any advantage the full model shows over self-training then belongs specifically to within-writer structure. If no such advantage appears, we report that the slip/gap separation is useful only for diagnosis.
Why not fix slip rates by hand, as Brunskill did? The file shows error classes with very different base rates; punctuation alone makes up roughly 30 to 39% of QALB errors. A single hand-set rate cannot fit classes that different. The synthetic experiment tests this directly, by comparing parameter recovery under fixed versus learned rates.
Proposed Experiments
The decisive correction experiment is QALB-2015 L2 test, scored with the M2 scorer (Maximum Match scorer, which compares system edits against gold edits). We report $F_1$ for comparability with the shared task and $F_{0.5}$ for comparability with recent work. We report results both with and without punctuation, because punctuation dominates QALB errors and can hide morphological gains (The First QALB Shared Task on Automatic Text Correction for Arabic).
As a secondary check, we score QALB-2015 native and ZAEBUC to confirm that adaptation does not hurt native-like text. On ZAEBUC we would expect the mastery variable to matter less, since those essays come from native university students.
The decisive diagnostic experiment splits each learner essay in half. We infer mastery posteriors from the first half only. We then test whether those posteriors predict which feature-level errors, as typed by ARETA against the gold corrections, occur in the second half. Performance is measured as AUC, in the spirit of COMMAND's predictive evaluation (Joint Discovery of Skill Prerequisite Graphs and Student Models). We compare against two predictors: raw first-half error-type frequencies, and the no-mastery model. Mastery posteriors must beat raw frequencies. Otherwise the latent structure is just a smoothed count and the slip/gap distinction has not been shown to be real.
On the synthetic benchmark, we report mastery-recovery accuracy and the correlation between recovered and true rates, as essay length varies. This establishes how much text per writer the method needs before being applied to real learners.
References
- The Second QALB Shared Task on Automatic Text Correction for Arabic
- Automatic Error Type Annotation for Arabic
- Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Study
- Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigation
- Identifying and analyzing ‘noisy’ spelling errors in a second language corpus
- Estimating Prerequisite Structure From Noisy Data
- Joint Discovery of Skill Prerequisite Graphs and Student Models
- The Columbia System in the QALB-2014 Shared Task on Arabic Error Correction
- Interactive Word Completion for Morphologically Complex Languages
- The First QALB Shared Task on Automatic Text Correction for Arabic