Hybrid Oracle: Making Use of Ambiguity in Transition-based Chinese Dependency Parsing

Xuancheng Ren, Xu SunarXiv 2017 · arXiv

Abstract

Transition-based dependency parsers often struggle with "spurious ambiguity," where multiple action sequences lead to the same gold-standard tree. Traditional "Standard Oracles" resolve this by enforcing a fixed path, which can be suboptimal for left-branching languages like Chinese. This paper proposes a Hybrid Oracle that leverages all valid transitions during training. By using a refined loss function and dynamic exploration, the model achieves significant improvements in accuracy, particularly for long-range dependencies and complex linguistic structures.

Introduction

In transition-based dependency parsing, a sentence is processed through a sequence of actions such as shift, left-arc, and right-arc. During training, an "oracle" guides the model by indicating the correct next action based on the gold-standard parse tree.

However, these systems frequently encounter spurious ambiguity: multiple valid action sequences can result in the same final tree. For instance, if a word has dependents on both sides, the parser could attach them in different orders. Traditionally, a "Standard Oracle" resolves this with a hardcoded preference, usually forcing left dependents to be attached as early as possible. While effective for English, this rigid path can lead to cascading errors if the model deviates early on. For heavily left-branching languages like Chinese, forcing early left attachments is particularly problematic as the left side of the parse is often the most complex and ambiguous.

The Hybrid Oracle Approach

The authors propose a Hybrid Oracle for arc-standard transition systems. Instead of selecting a single "best" transition, the Hybrid Oracle identifies all mathematically valid transitions that can lead to the gold parse tree. This multi-path awareness is integrated into neural network training through two mechanisms:

  1. Refined Loss Function: The target for cross-entropy loss becomes a uniform probability distribution over all valid actions. If both shift and left-arc are correct, the target vector assigns a 0.5 probability to each.
  2. Dynamic Exploration: During training, the system randomly samples one of the valid transitions. This allows the model to explore different correct parsing paths for the same sentence across epochs, acting as a form of data augmentation and improving generalization.

Experimental Setup

The approach was evaluated on two standard Chinese dependency parsing benchmarks: CTB5 (Chinese Treebank 5.0) and CoNLL17.

Dataset statistics for CTB5 and CoNLL17, highlighting the prevalence of ambiguous tokens and sentences.

As shown in the table above, both datasets contain a high degree of structural ambiguity. In CoNLL17, for instance, nearly 56.54% of tokens can be parsed via multiple sequences, providing a significant opportunity for the Hybrid Oracle to improve upon the rigid Standard Oracle.

Results and Analysis

The Hybrid Oracle consistently outperformed the Standard Oracle across all metrics. On the CTB5 test set, the Unlabeled Attachment Score (UAS) improved from 85.03 to 85.68, while the Unlabeled Exact Match (UEM) jumped from 31.20 to 33.50. Similar gains were observed on CoNLL17, where UAS improved by 0.83 points.

Comparison of parsing accuracy (UAS) between the Standard and Hybrid Oracles across different dependency distances.

A key finding is that the Hybrid Oracle is particularly effective at improving the recall of long-range dependency arcs. As illustrated in the performance plot, the hybrid approach (red line) maintains higher accuracy across various dependency distances, especially for left-branching structures (negative x-axis values). Since over 72% of dependents in CoNLL17 are on the left, the Hybrid Oracle's ability to delay difficult decisions via shift until more context is available proves highly beneficial.

Conclusion

By embracing rather than suppressing ambiguity, the Hybrid Oracle allows neural parsers to learn more robust representations. The model is no longer forced to follow an arbitrary tie-breaking rule and can instead choose transitions it is most confident in based on the linguistic context. While the absolute scores in this study were achieved using a vanilla architecture to isolate the oracle's impact, the relative improvements suggest that this approach is a powerful tool for parsing languages with complex branching structures. Future work will explore whether these benefits extend to other left-branching languages cross-lingually.

Summary updated

Abstract

In the training of transition-based dependency parsers, an oracle is used to predict a transition sequence for a sentence and its gold tree. However, the transition system may exhibit ambiguity, that is, there can be multiple correct transition sequences that form the gold tree. We propose to make use of the property in the training of neural dependency parsers, and present the Hybrid Oracle. The new oracle gives all the correct transitions for a parsing state, which are used in the cross entropy loss function to provide better supervisory signal. It is also used to generate different transition sequences for a sentence to better explore the training data and improve the generalization ability of the parser. Evaluations show that the parsers trained using the hybrid oracle outperform the parsers using the traditional oracle in Chinese dependency parsing. We provide analysis from a linguistic view. The code is available at https://github.com/lancopku/nndep .
BibTeX
@misc{ren2017hybrid,
  author = {Xuancheng Ren and Xu Sun},
  title = {{Hybrid Oracle: Making Use of Ambiguity in Transition-based Chinese Dependency Parsing}},
  year = {2017},
  eprint = {1711.10163},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1711.10163},
}
Figure