Abstract
We connect two scenarios in structured learning: adapting a parser trained on one corpus to another annotation style, and projecting syntactic annotations from one language to another. We propose quasi-synchronous grammar (QG) features for these structured learning tasks. That is, we score a aligned pair of source and target trees based on local features of the trees and the alignment. Our quasi-synchronous model assigns positive probability to any alignment of any trees, in contrast to a synchronous grammar, which would insist on some form of structural parallelism.
In monolingual dependency parser adaptation, we achieve high accuracy in translating among multiple annotation styles for the same sentence. On the more difficult problem of cross-lingual parser projection, we learn a dependency parser for a target language by using bilingual text, an English parser, and automatic word alignments. Our experiments show that unsupervised QG projection improves on parses trained using only high-precision projected annotations and far outperforms, by more than 35% absolute dependency accuracy, learning an unsupervised parser from raw target-language text alone. When a few target-language parse trees are available, projection gives a boost equivalent to doubling the number of target-language trees.
The Challenge of Syntactic Diversity
Building a high-quality dependency parser—a tool that determines the grammatical structure of a sentence—usually requires a massive amount of hand-annotated data. This creates two major bottlenecks in the field of natural language processing:
- Annotation Inconsistency: Different linguistic projects often adopt different "styles" or conventions for representing the same sentence. For example, should a preposition be the head of a noun, or vice versa? If a researcher has 10,000 sentences in Style A but requires Style B for their application, they typically cannot mix the datasets without significant loss in performance.
- The Language Gap: For many of the world's languages, annotated "treebanks" are scarce or non-existent. While we often have access to "bitexts" (translated pairs of sentences, such as UN proceedings), transferring the syntactic knowledge from a high-resource language like English to a low-resource language remains difficult.
Previous attempts to solve these issues often relied on "Synchronous Grammars," which are mathematically rigid and assume that the grammar of two languages (or styles) must mirror each other perfectly. When they don't—which is common in natural language—these models become computationally expensive or fail to generalize.
Methods: Quasi-Synchronous Grammar (QG)
The authors propose using Quasi-Synchronous Grammar (QG). The "Quasi" prefix is critical: it does not insist on a perfect structural match between source and target. Instead, it scores a target tree based on how well it aligns with a source tree using local features.
- Configurations: Rather than requiring a parent-child pair in English to remain a parent-child pair in the target language, the model looks at "configurations." It asks: "If these two words are parent and child in the source, what is their relationship in the target?" They might be siblings, or the head might be swapped.
- Adaptation Model: A conditional model $p(t | w, w', t', a)$ predicts a target tree ($t$) given the target words ($w$), source words ($w'$), source tree ($t'$), and word alignments ($a$).
- Unsupervised Projection: For languages with no trees, the researchers use a generative model (the Dependency Model with Valence or DMV) and train it using Expectation-Maximization (EM). Crucially, they condition the EM process on the source-language tree, using it as a "soft" hint to guide the learner toward the correct structure.
- Efficiency: By using existing word alignments (like IBM Model 4) to restrict the search space, the model maintains a complexity of $O(n^3 k^2)$, making it nearly as fast as standard monolingual parsing.
Key Results and Insights
The researchers evaluated their approach across monolingual adaptation (using the Penn Treebank) and cross-lingual projection (using German TIGER, Spanish Cast3LB, and English-Chinese ECTB treebanks).
- Unsupervised Success: In zero-resource scenarios, the QG model achieved 68.5% accuracy for German and 64.8% for Spanish. This far outperformed standard unsupervised EM (which hit only ~30% and ~25% respectively) and even beat "hard" projection methods that literally copy links from English.
- The "Data Doubler" Effect: One of the most striking findings is that having access to a source-language tree is roughly equivalent to doubling the amount of manually annotated data in the target language. For example, a parser trained on 100 Chinese trees with English projection features performed as well as a parser trained on 200 Chinese trees alone.
- Robustness to Divergence: Because QG doesn't require isomorphism, it naturally handles linguistic differences, such as English being head-initial while another language might be head-final, or differing ways of handling auxiliary verbs.
Conclusion
This framework demonstrates that source-language parsers can act as powerful "soft" guides for target-language learning. By allowing for "sloppy" alignments rather than strict structural matches, QG features significantly boost accuracy in low-resource settings and effectively bridge the gap between disparate annotation styles.
Summary updated