Abstract
While syntactic parsing for major languages like English and Chinese has reached high levels of maturity, Vietnamese parsing continues to struggle. This paper identifies that the performance bottleneck isn't just a lack of data, but rather deep-seated linguistic ambiguities—specifically around word order and part-of-speech (POS) tagging—that current state-of-the-art neural parsers fail to resolve.
Introduction
Syntactic parsing is a foundational task in natural language processing, yet Vietnamese parsers consistently underperform compared to their English counterparts. Researchers have long debated whether this is a "low-resource" problem or if there is something inherently complex about Vietnamese grammar that breaks standard algorithms. This study performs a diagnostic on Vietnamese parsing, evaluating top-tier models to identify frequent error types and root causes.
Experimental Methodology
The researchers employed a multi-stage design to evaluate performance:
- Benchmarking: Testing six representative constituency parsers, ranging from classic PCFG models (Stanford-Unlex) to modern transition-based neural models (RNNGs).
- Automatic Error Classification: Adapting a tool by Kummerfeld et al. to categorize errors into types like "VP attachment" or "NP attachment."
- Sensitivity Analysis: Observing performance changes with varying training data sizes and comparing predicted vs. "gold" (perfect) POS tags.
- Manual Investigation: Inspecting 100 parsed sentences to identify specific linguistic "traps" that lead parsers astray.
The study utilized the Vietnamese Treebank (VTB), containing approximately 10,377 sentences with gold-standard word segmentation to isolate parsing performance from segmentation errors.
The Data Plateau
A significant finding of the study was that increasing training data beyond a certain point yielded diminishing returns. As shown in Figure 1, performance gains nearly stalled when moving from 6,337 to 8,337 training sentences across all models. This suggests that the accuracy gap cannot be solved by simply collecting more data of the same type; the bottleneck is likely structural.
Error Analysis and POS Impact
The top performer, RNNGs-G (Generative Recurrent Neural Network Grammar), achieved an F1 score of 72.94. While respectable, this remains significantly lower than the 90+ scores typical for English. While using "Gold" POS tags improved F1 scores by roughly 5.26 points, the most common attachment errors (VP and NP) persisted, suggesting the current tagset (33 tags) may be too coarse to resolve structural ambiguities.
In English, Prepositional Phrase (PP) attachment is often the primary challenge. In Vietnamese, however, Verb Phrase (VP) attachment is the most frequent error. This is largely due to "linguistic camouflage": Vietnamese lacks inflectional morphemes, meaning a verb can look identical whether it is a main verb or a modifier for a noun. This creates "structural mirrors" where a Noun Phrase (NP) can be mistaken for a simple Subject-Verb sentence.
Structural Ambiguity: The "Cho" Case Study
Manual analysis revealed that 63% of attachment errors were caused by inherent grammatical ambiguities. A prime example is the word cho, which can function as a conjunction ("so that") introducing a clause or a preposition ("for") introducing a noun phrase.
As illustrated in Figure 6, parsers often fail to use context to distinguish between these roles, leading to incorrect structural assignments, such as mistaking a subordinate clause (SBAR) for a prepositional phrase (PP).
Conclusion and Future Directions
The study concludes that Vietnamese parsing errors are deeply rooted in structural ambiguity rather than just data scarcity. The authors suggest several future directions to bridge the performance gap:
- Symbol Splitting: Refining the POS tagset to help models distinguish between different structural roles (e.g., splitting conjunction tags into more specific sub-tags).
- Hybrid Models: Combining the global context strengths of neural networks with the explicit, symbolic distinctions found in older parsers.
- Language-Specific Features: Incorporating features that account for Vietnamese word order, where modifiers typically follow the head noun, to resolve NP vs. sentence ambiguities.
Summary updated