Juman++: A Morphological Analysis Toolkit for Scriptio Continua

Arseny Tolmachev, Daisuke Kawahara, Sadao KurohashiEMNLP 2018

TL;DR

This paper introduces Juman++ V2, a significantly faster and more accurate morphological analyzer for languages without explicit word boundaries, like Japanese. It achieves state-of-the-art results while also providing tools to aid in the development and improvement of such analyzers, addressing the costly annotation bottleneck.

What Problem Does This Paper Address?

Processing languages like Japanese, where words are not separated by spaces (scriptio continua), requires morphological analysis – essentially, breaking down sentences into their smallest meaningful units (morphemes) and identifying their grammatical roles. While existing tools can achieve high accuracy on specific domains like newspapers, their performance degrades on more diverse, open-domain data (e.g., web text). Improving accuracy in these scenarios typically demands extensive, costly manual annotation by experts, which is a major bottleneck. The previous Juman++ V1 analyzer, while accurate, was prohibitively slow for practical use. This paper aims to create a faster, more accurate analyzer and provide tools to reduce the reliance on expensive manual annotation for further improvements.

Methods

Juman++ V2 is a lattice-based morphological analyzer that uses a combination of a linear model and a recurrent neural network language model (RNNLM) to score potential segmentations. The core innovation lies in its efficiency improvements and the introduction of a "trimmed beam search" strategy.

  • Trimmed Beam Search: Instead of considering all possible paths in the lattice (which can be computationally explosive), Juman++ V2 employs "left global beam" and "right global beam" techniques. These methods prune less promising paths early in the search, significantly reducing the computational load without sacrificing accuracy. This is a key departure from the "full beam" search used in V1.
  • Rewrite and Optimization: Juman++ V2 is a complete rewrite of V1 in modern C++, designed for both standalone use and as an embeddable library. It's language and segmentation standard independent.
  • Development Tools: The toolkit includes:
  • A morphological analyzer developer tool that uses unannotated data to highlight problematic dictionary entries or grammar rules by comparing analysis results from different beam search configurations.
  • A partial annotation tool that simplifies the process of reviewing and correcting analysis errors, allowing annotators to choose between candidate analyses or provide corrections, generating partially annotated data for training.

Datasets

The paper uses the following datasets for training and evaluation:

  • Jumandic-based corpora: The standard benchmark for Japanese morphological analysis.
  • Kyoto University (KU) corpus: Used for training and evaluation.
  • KWDLC corpus: Used for training and evaluation.
  • Web corpus: A 50k sentence corpus used for speed comparison.
  • Augmented data: For annotation experiments, data was augmented by removing common case markers ("ga", "ha", "wo") to improve robustness.

No new benchmarks were introduced, but the paper focuses on improving performance on existing ones.

Key Insights and Ideas

  • Trimmed beam search is surprisingly effective: The paper demonstrates that significantly pruning the search space through global beams doesn't hurt accuracy and can even reveal issues with the full beam analysis in certain cases. This suggests that many paths considered by a full beam search are unlikely to be correct.
  • Diffs reveal more than just training data issues: Sentences where the trimmed and full beam searches disagree ("diffs") are not solely indicative of problems in the training data. They can also highlight limitations in the analysis dictionary or even fundamental ambiguities in the language that are difficult to resolve.
  • Partial annotation is a viable path to improvement: The partial annotation tool, by allowing annotators to select between candidate analyses, significantly speeds up the annotation process compared to full manual annotation, making it more practical for domain adaptation and improving accuracy.
  • Speed and accuracy are not mutually exclusive: Juman++ V2 proves that it's possible to achieve both high accuracy and practical speeds, overcoming a major limitation of its predecessor.

Key Results

  • Speed: Juman++ V2 (RNN version) is over 250 times faster than Juman++ V1. It is only 20% slower than the traditional JUMAN analyzer, despite having a more complex model.
  • Accuracy: Juman++ V2 achieves new state-of-the-art (SOTA) accuracy on Jumandic-based corpora, outperforming previous SOTA models like V1 RNN. For example, V2 RNN achieves an F1 score of 99.51 on segmentation and POS tagging for the KU corpus.
  • Annotation Experiment: An inter-annotator agreement of 0.819 was achieved with the partial annotation tool, and annotators were able to review a significant number of sentences in a short period.

Limitations and Future Work

  • Limitations:
    • The current implementation focuses on Japanese and the Jumandic standard, though the core library is language-independent.
  • The "diffs" identified by the developer tool can sometimes be difficult even for human annotators to resolve, suggesting complex linguistic phenomena or data issues.
  • Future Work:
    • Develop a Unidic version of Juman++ V2.
  • Use the developed tools to annotate readings for Jumandic-based corpora, which are currently arbitrary.
  • Collect and release a partially annotated web corpus.
  • Continue collecting partially annotated data for further model improvements.

Summary updated

BibTeX
@inproceedings{tolmachev2018juman,
  author = {Arseny Tolmachev and Daisuke Kawahara and Sadao Kurohashi},
  title = {{Juman++: A Morphological Analysis Toolkit for Scriptio Continua}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure