Kanji Workbook: A Writing-Based Intelligent Tutoring System for Learning Proper Japanese Kanji Writing Technique with Instructor-Emulated Assessment

Paul Taele, Jung In Koh, Tracy HammondAAAI 2020 · DOI

Introduction

Learning Japanese Kanji presents a significant challenge for non-native speakers due to the complexity of character structures and the pedagogical importance of writing technique. In traditional settings, instructors emphasize stroke order and direction as foundational for both legibility and memorization. However, providing real-time, personalized feedback in large classrooms is often unfeasible. While many mobile applications exist for Kanji practice, they typically focus on the final visual output (Optical Character Recognition) rather than the temporal process of writing.

To address this gap, researchers developed Kanji Workbook, an intelligent tutoring system (ITS) designed to emulate human instructor assessment by evaluating the specific techniques used during the writing process.

The Ten Assessment Metrics

The system's core innovation lies in its ten diagnostic metrics, derived from qualitative studies of language instructors. These metrics are divided into three categories:

  1. Structure: Ensures the correct number of strokes are present and correctly paired (Stroke Match, Valid, and Exist).
  2. Technique: Validates the temporal sequence and path of the pen (Stroke Order and Stroke Direction).
  3. Precision: Measures spatial accuracy and "fluency" (Stroke Edit, Length, Closeness, Stroke Speed, and Symbol Speed).
The system highlights specific strokes to provide granular feedback on technique, such as the horizontal stroke shown here.

By monitoring "edits" (undos) and stroke speed, the system can detect student hesitation, serving as a proxy for a lack of confidence or mastery.

Technical Implementation

The Kanji Workbook processes student input through a multi-stage pipeline:

  • Preprocessing: Input is normalized via resampling to 64 points, scaled to a 250px bounding box, and translated to a Cartesian origin to ensure consistency across devices.
  • Algorithms: The system utilizes the Hausdorff distance metric to pair student strokes with expert templates. For directional assessment, Euclidean distance compares the start and end points of each stroke.
  • Scoring: Drawing from the $P and $1 gesture classifiers, the system provides a three-star rating for each metric, offering immediate diagnostic feedback.
Example of the system isolating a vertical hook stroke for individual assessment of precision and length.

Evaluation and Results

The system was evaluated over a full academic year with 94 university students. The "case" group (using the app) was compared against a "control" group using traditional paper-based methods.

The results demonstrated a statistically significant improvement in learning outcomes:

  • Grade Improvement: First-year students using the Kanji Workbook scored an average of 9.10 points higher than the control group.
  • Overall Performance: Across all proficiency levels, the app users outperformed the control group by 6.22 points (p = 0.046).

Qualitative feedback indicated that students found the three-star feedback system rewarding, although some noted that the automated grading could be stricter than a human instructor.

Conclusion

Kanji Workbook successfully bridges the gap between automated character recognition and pedagogical instruction. By focusing on the process of writing, the system provides students with the real-time corrections necessary to build proper habits. Future work will focus on refining the user interface, tuning assessment thresholds for better alignment with human leniency, and expanding the curriculum to include Chinese characters and advanced Japanese levels.

Summary updated

BibTeX
@inproceedings{taele2020kanji,
  author = {Paul Taele and Jung In Koh and Tracy Hammond},
  title = {{Kanji Workbook: A Writing-Based Intelligent Tutoring System for Learning Proper Japanese Kanji Writing Technique with Instructor-Emulated Assessment}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i08.7053},
}
Figure