A neural parser as a direct classifier for head-final languages

Hiroshi Kanayama, Masayasu Muraoka, Ryosuke KohitaACL 2018

TL;DR

This paper introduces a novel neural network-based dependency parser specifically designed for head-final languages like Japanese. By directly classifying the head word of a dependent from a limited set of candidates, it simplifies the model, enhances interpretability, and achieves state-of-the-art results on Japanese dependency parsing tasks.

What Problem Does This Paper Address?

Dependency parsing aims to uncover the grammatical relationships between words in a sentence, forming a tree-like structure. Traditional methods often rely on complex transition-based or graph-based algorithms, which can be difficult to interpret and may struggle with languages that have different word orders. Japanese, being a head-final language (where the head of a phrase typically comes last), presents unique challenges. Existing multilingual parsers often perform poorly on Japanese, as demonstrated by the low scores in the 2017 Universal Dependencies shared task. This paper addresses the need for a more effective and interpretable parsing approach tailored to the specific characteristics of head-final languages like Japanese.

Methods

The core innovation of this paper lies in its departure from traditional parsing algorithms. Instead of complex state transitions or graph constructions, the proposed neural parser directly acts as a classifier. For each dependent word (or phrasal unit, PU), it identifies a limited set of potential head words (candidates) to its right, leveraging grammatical rules derived from the "Triplet/Quadruplet Model." The neural network then predicts the most likely head from these candidates. This approach simplifies the model by focusing on a direct classification task. The model utilizes distributed representations (embeddings) for words and their parts-of-speech, along with features capturing relationships between the modifier and candidate PUs (e.g., number of postpositions, commas, and distance). The final output is a probability distribution over the candidate heads, determined via a softmax function. For sentence-level parsing, a beam search is employed to construct a projective dependency tree, ensuring that the predicted structure adheres to the language's constraints.

Datasets

The primary dataset used for training and evaluation is the EDR Japanese Corpus. For comparison with existing work, the authors also utilize the UD Japanese-GSD (Universal Dependencies Japanese-GSD) test data.

Key Insights and Ideas

  • Head-final simplification: The head-final property of Japanese allows for a significant simplification of the parsing problem, enabling a direct classification approach rather than complex sequential decision-making.
  • Interpretability through direct classification: By directly selecting the head from a limited set of candidates, the model's output is more intuitive and easier to interpret compared to the opaque decision-making of transition-based parsers.
  • Grammatical knowledge integration: The model effectively incorporates grammatical knowledge (via the Triplet/Quadruplet Model) to restrict candidate heads, reducing errors without requiring additional annotated corpora.
  • Distributed representations: The use of distributed word embeddings allows the model to handle a wider range of vocabularies and capture nuanced relationships that might be missed by traditional feature-based methods.
  • Importance of commas: Ablation studies revealed that commas play a surprisingly significant role in dependency parsing, contributing to a nearly 2-point accuracy drop when removed.

Key Results

The proposed neural parser achieved state-of-the-art results on the EDR Japanese corpus, outperforming a logistic regression baseline and other models when trained on sufficient data (120k+ sentences). Specifically, with 160k training sentences, the neural model achieved an accuracy of 89.31%, surpassing the logistic regression model. When evaluated on the UD Japanese-GSD test set, the model achieved a 94.03% UAS score, outperforming other systems in the 2017 Shared Task, even when tokenization was not gold-standard. The neural model also demonstrated a significant speed advantage, converging in seconds on a GPU compared to hours for the logistic regression model on a CPU.

Limitations and Future Work

The authors acknowledge that their model does not explicitly handle coordination relationships, which they plan to address in future work by incorporating more lexical knowledge and broader contextual information. They also suggest integrating LSTM models to capture more contextual information for further improvement. While the model performs well on Japanese, extending it to other head-final languages like Korean and Tamil is a potential future direction, though challenges related to corpus consistency and data availability exist.

Summary updated

BibTeX
@inproceedings{kanayama2018neural,
  author = {Hiroshi Kanayama and Masayasu Muraoka and Ryosuke Kohita},
  title = {{A neural parser as a direct classifier for head-final languages}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2018},
}
Figure