TL;DR
This paper tackles the challenge of automatically assigning semantic roles to Chinese sentences, a crucial step for understanding sentence meaning. They demonstrate that surprisingly good results can be achieved with a small training set by adapting existing English semantic parsing techniques and a newly ported Chinese syntactic parser, finding that Chinese is, in fact, easier to parse semantically than English due to stricter word order.
What Problem Does This Paper Address?
Understanding the meaning of a sentence goes beyond just its grammatical structure. Semantic parsing aims to identify the "who did what to whom, where, and when" by assigning thematic roles (like AGENT, THEME, LOCATION) to sentence constituents. While significant progress has been made for English, applying these methods to other languages, especially those with different grammatical structures like Chinese, presents challenges. Key questions include: how well do existing algorithms perform on languages with limited labeled data, and what role do language-specific features play in semantic role assignment? This paper specifically addresses these questions for Chinese.
Methods
The authors adapt an SVM-based semantic parsing algorithm originally developed for English by Pradhan et al. (2003). Their approach treats semantic parsing as a classification problem where each constituent in a syntactic parse tree is assigned a semantic role label (or "NULL" if it's not an argument).
Key methodological contributions include:
- Chinese Corpus Creation: They created a small, hand-labeled corpus of 1100 Chinese sentences based on principles from English and Chinese PropBanks.
- Feature Engineering: They adapted features commonly used in English semantic parsing, such as syntactic phrase type, head word, governing category, and the path in the parse tree connecting a constituent to the verb. They also explored language-specific features like the position of constituents relative to the verb.
- Collins Parser Porting: A significant effort was made to port the state-of-the-art Collins parser (a lexicalized statistical parser) to Chinese. This involved developing head rules for the Chinese Treebank grammar and adapting punctuation handling. This ported parser was then used to generate automatic syntactic parses for their experiments.
Datasets
- Chinese Training/Test Set: A custom corpus of 1100 Chinese sentences was created by selecting sentences containing 10 specific Chinese verbs. This corpus was split into a training set (1025 sentences) and a test set (113 sentences).
- Penn Chinese Treebank (CTB): Used for extracting sentences for the custom corpus and for training the Collins parser.
- Penn English Propbank: Used as a basis for the Chinese corpus annotation principles and for creating a comparable English dataset.
Key Insights and Ideas
- Small Data Sufficiency: Good semantic parsing results for Chinese can be achieved with a surprisingly small training set (around 1100 sentences). This is a crucial finding for low-resource languages.
- Cross-Lingual Feature Transfer: Features that work well for English semantic parsing are largely effective for Chinese, suggesting a degree of universality in semantic role assignment cues.
- Chinese Syntactic Parser Performance: The ported Collins parser achieved the best reported performance on Chinese syntactic parsing at the time of publication.
- Chinese is Easier for Semantic Parsing: Counterintuitively, the paper finds that semantic parsing is significantly easier in Chinese than in English. This is attributed to:
- Strict Adjunct Ordering: Chinese has much stricter word order constraints for adjuncts (like time, location, manner) compared to English. This makes them easier to identify and label.
- Passive Voice Rarity: The marked passive voice, which is a significant feature in English semantic parsing, is very rare in Chinese, simplifying the analysis.
- Lexicalized Features are Key: Features like the target verb and head word are highly important for accurate semantic role assignment.
Key Results
- Seen Verbs: The combined feature set achieved an F-score of 83.1% on seen verbs.
- Unseen Verbs: The average F-score on unseen verbs was 58.3%, with significant variation across verbs (e.g., F-score of 11.3% for "出现/emerge" vs. 80.8% for "发表/publish").
- Automatic Parses: Using automatic syntactic parses from the ported Collins parser reduced the F-score by 6.4% compared to using hand-corrected parses, resulting in an F-score of 77.6%.
- English vs. Chinese: The Chinese system achieved an F-score of 71.5% on a comparable English dataset, significantly lower than its performance on Chinese. The difference was largely attributed to adjuncts, where Chinese achieved an F-score of 78.6% compared to 36.7% for English.
Limitations and Future Work
- Unseen Verb Performance: The performance on unseen verbs is still a significant challenge, particularly for verbs with idiosyncratic syntactic behavior.
- Data Sparsity: While small data is sufficient for good results, the sparsity of features like parse paths and head words remains a challenge.
- Future Work: The authors suggest further investigation into the impact of different feature sets, exploring more sophisticated parsing techniques, and expanding the annotated corpus to cover a wider range of verbs and linguistic phenomena. They also highlight the need for more systematic comparisons between languages to understand cross-lingual transferability and language-specific challenges in NLP.
Summary updated