Abstract
Transition-based dependency parsers often struggle with "spurious ambiguity," where multiple action sequences lead to the same gold-standard tree. Traditional "Standard Oracles" resolve this by enforcing a fixed path, which can be suboptimal for left-branching languages like Chinese. This paper proposes a Hybrid Oracle that leverages all valid transitions during training. By using a refined loss function and dynamic exploration, the model achieves significant improvements in accuracy, particularly for long-range dependencies and complex linguistic structures.
Introduction
In transition-based dependency parsing, a sentence is processed through a sequence of actions such as shift, left-arc, and right-arc. During training, an "oracle" guides the model by indicating the correct next action based on the gold-standard parse tree.
However, these systems frequently encounter spurious ambiguity: multiple valid action sequences can result in the same final tree. For instance, if a word has dependents on both sides, the parser could attach them in different orders. Traditionally, a "Standard Oracle" resolves this with a hardcoded preference, usually forcing left dependents to be attached as early as possible. While effective for English, this rigid path can lead to cascading errors if the model deviates early on. For heavily left-branching languages like Chinese, forcing early left attachments is particularly problematic as the left side of the parse is often the most complex and ambiguous.
The Hybrid Oracle Approach
The authors propose a Hybrid Oracle for arc-standard transition systems. Instead of selecting a single "best" transition, the Hybrid Oracle identifies all mathematically valid transitions that can lead to the gold parse tree. This multi-path awareness is integrated into neural network training through two mechanisms:
- Refined Loss Function: The target for cross-entropy loss becomes a uniform probability distribution over all valid actions. If both
shiftandleft-arcare correct, the target vector assigns a 0.5 probability to each. - Dynamic Exploration: During training, the system randomly samples one of the valid transitions. This allows the model to explore different correct parsing paths for the same sentence across epochs, acting as a form of data augmentation and improving generalization.
Experimental Setup
The approach was evaluated on two standard Chinese dependency parsing benchmarks: CTB5 (Chinese Treebank 5.0) and CoNLL17.
As shown in the table above, both datasets contain a high degree of structural ambiguity. In CoNLL17, for instance, nearly 56.54% of tokens can be parsed via multiple sequences, providing a significant opportunity for the Hybrid Oracle to improve upon the rigid Standard Oracle.
Results and Analysis
The Hybrid Oracle consistently outperformed the Standard Oracle across all metrics. On the CTB5 test set, the Unlabeled Attachment Score (UAS) improved from 85.03 to 85.68, while the Unlabeled Exact Match (UEM) jumped from 31.20 to 33.50. Similar gains were observed on CoNLL17, where UAS improved by 0.83 points.
A key finding is that the Hybrid Oracle is particularly effective at improving the recall of long-range dependency arcs. As illustrated in the performance plot, the hybrid approach (red line) maintains higher accuracy across various dependency distances, especially for left-branching structures (negative x-axis values). Since over 72% of dependents in CoNLL17 are on the left, the Hybrid Oracle's ability to delay difficult decisions via shift until more context is available proves highly beneficial.
Conclusion
By embracing rather than suppressing ambiguity, the Hybrid Oracle allows neural parsers to learn more robust representations. The model is no longer forced to follow an arbitrary tie-breaking rule and can instead choose transitions it is most confident in based on the linguistic context. While the absolute scores in this study were achieved using a vanilla architecture to isolate the oracle's impact, the relative improvements suggest that this approach is a powerful tool for parsing languages with complex branching structures. Future work will explore whether these benefits extend to other left-branching languages cross-lingually.
Summary updated