Abstract
Learning Japanese vocabulary is a challenge for learners from Roman alphabet backgrounds due to script differences. Japanese combines syllabaries like hiragana with kanji, which are logographic characters of Chinese origin. Kanji are also complicated due to their complexity and volume. Keyword mnemonics are a common strategy to aid memorization, often using the compositional structure of kanji to form vivid associations. Despite recent efforts to use large language models (LLMs) to assist learners, existing methods for LLM-based keyword mnemonic generation function as a black box, offering limited interpretability. We propose a generative framework that explicitly models the mnemonic construction process as driven by a set of common rules, and learn them using a novel Expectation-Maximization-type algorithm. Trained on learner-authored mnemonics from an online platform, our method learns latent structures and compositional rules, enabling interpretable and systematic mnemonics generation. Experiments show that our method performs well in the cold-start setting for new learners while providing insight into the mechanisms behind effective mnemonic creation.
Introduction
For students learning Japanese from a Roman alphabet background, Kanji represents a significant hurdle. With thousands of characters and complex structures, learners often rely on keyword mnemonics: breaking a character into its components (radicals) and weaving them into a narrative. For example, the character for "rest" (休) is composed of "person" (人) and "tree" (木), leading to the mnemonic: "A person is resting by a tree."
While Large Language Models (LLMs) excel at narrative generation, they typically function as "black boxes." They do not explain the logic behind specific narrative choices, nor do they easily adapt to individual learner preferences. This research addresses the lack of interpretability and systematic generation in AI-assisted Kanji learning by proposing a rule-driven generative framework.
Methodology
The proposed framework treats mnemonic construction as a systematic process governed by latent rules. It utilizes a hybrid approach combining two LLMs with a statistical algorithm.
Latent Trait Modeling
Borrowing from psychometrics, the researchers use a 1-Parameter Logistic Item Response Theory (IRT) model. This model calculates the probability of a specific rule being used based on two factors:
- Learner Affinity ($h_{jk}$): A student's preference for certain story types (e.g., humorous or vivid).
- Kanji Compatibility ($g_{ik}$): How well a specific character's structure fits a particular rule (e.g., ease of personification).
The EM-type Algorithm
The system is trained using an Expectation-Maximization (EM) algorithm:
- E-Step (Rule Assignment): An open-weights model (Llama-3.2 3B) evaluates which rules most likely apply to existing human-authored mnemonics from the Koohii Kanji platform.
- M-Step (Parameter & Rule Update): The system updates learner and Kanji latent variables. Simultaneously, GPT-4o refines the rules themselves to ensure they are distinct and interpretable.
- Fine-tuning: Finally, the Llama model is instruction-tuned to strictly follow these learned rules during generation.
Experimental Results
The model was trained and tested on data from Koohii Kanji, involving over 11,000 learners and 2,200 unique Kanji. The EM-based method was compared against standard Supervised Fine-Tuning (SFT) and Zero-Shot (ZS) baselines.
As shown in the results, the EM method achieved a 58.1% win rate over the SFT baseline in head-to-head evaluations using an LLM-as-a-judge (Prometheus). It also demonstrated consistent gains in BERTScore and ROUGE metrics, indicating that its outputs are closer to human-authored mnemonics in both semantic meaning and vocabulary. Furthermore, the model achieved a 60.58% compliance rate in following the learned rules.
Discussion and Conclusion
The EM process successfully distilled ten distinct, interpretable rules, such as "utilizing vivid imagery" and "employing anthropomorphism." Unsupervised clustering revealed that Kanji naturally group into "personalities" based on which rules suit them best—some are inherently "logical," while others are better suited for "vivid imagery."
Crucially, the method performs well in "cold-start" scenarios for new learners by using population-level averages for learner traits. While future work is needed to confirm long-term retention through human trials and explore real-time adaptive personalization, this framework provides a transparent and systematic approach to personalized language learning.
Summary updated