TL;DR
This paper introduces the creation of a large-scale Chinese corpus (1 million words) annotated with semantic dependencies, a richer form of linguistic annotation than traditional syntactic structures. The authors detail their novel tagging scheme, consistency improvement strategies, and a robust checking process, aiming to provide a valuable resource for advancing Chinese natural language processing.
What Problem Does This Paper Address?
While many natural language processing (NLP) corpora focus on syntactic knowledge (how words are arranged), understanding the meaning and relationships between words (semantic knowledge) is crucial for deeper language comprehension. For Chinese, in particular, syntactic rules are flexible, and semantic understanding is paramount. Existing Chinese corpora primarily offer syntactic annotations, with limited efforts in semantic dependency. This paper addresses the challenge of building a large, consistently annotated corpus for semantic dependencies in Chinese, a task made difficult by the inherent ambiguity of semantic relationships and the lack of established methodologies for such annotation at scale.
Methods
The authors developed a Semantic Dependency Net (SDN) approach, viewing words as the fundamental units of semantics and representing sentence meaning through a series of semantic dependencies between individual words. They designed a dependency grammar where each word can have a headword and a semantic relation.
Key methodological contributions include:
- Tagging Scheme: A set of 59 semantic relations, 9 syntactic relations, and 2 special relations, largely inspired by the HowNet lexical database but pared down for practicality.
- Tagging Tool: A computer-aided tool to assist annotators, providing an interactive interface for selecting headwords and relations, and visualizing the resulting dependency tree.
- Consistency Improvement: Strategies to address the inherent ambiguity in semantic annotation, including defining "congruence" as a measure of inter-annotator agreement and implementing a multi-round checking process.
- Checking Procedure: A rigorous five-round process involving selective manual checking and semi-automatic checking, where the tool flags potential errors based on HowNet role frames, custom error rules, and a simple semantic dependency model score.
Datasets
- Tsinghua Corpus: A portion of this corpus, originally annotated with semantic classes and derived from People's Daily news, served as the raw data.
- Newly Introduced Benchmark: The paper presents the SDN corpus, a 1,000,000-word scale corpus annotated with semantic dependencies. This is a novel contribution to Chinese NLP resources.
Key Insights and Ideas
- Semantic Dependency as a Richer Representation: The paper argues that semantic dependencies capture meaning more effectively than purely syntactic structures, especially for languages like Chinese with flexible syntax.
- Ambiguity is the Core Challenge: The authors highlight that semantic annotation is significantly more challenging than syntactic due to the higher degree of ambiguity and differing interpretations.
- HowNet as a Foundation: Leveraging the HowNet lexical database provided a strong starting point for defining semantic relations, but adaptation was necessary for corpus annotation.
- Consistency is Paramount: The emphasis on developing and measuring "congruence" (inter-annotator agreement) underscores the critical need for reliable and consistent annotations in semantic tasks.
- Semi-Automatic Checking is Essential: While manual annotation is necessary, a semi-automatic approach significantly speeds up the checking process by filtering potential errors, allowing human annotators to focus on complex cases.
Key Results
- SDN Corpus Creation: A 1,000,000-word Chinese corpus annotated with semantic dependencies was successfully built.
- High Congruency Achieved: After checking, the corpus achieved high inter-annotator agreement, with relation/sentence congruency reaching 96.24%/83.43% among three experts, indicating satisfactory quality.
- Annotation Speed: Manual annotation speed was approximately 1.15 hours per 100 sentences, while checking speed was significantly faster with semi-automatic methods (0.08 hours per 100 sentences).
Limitations and Future Work
- Inconsistencies Remain: Despite rigorous checking, the authors acknowledge that inconsistencies still exist and may become more apparent over time.
- Interface Development: A user-friendly, interactive interface for corpus investigation and maintenance is needed.
- Verb Role Frame Extraction: The paper suggests extracting verb role frames from the corpus, which could be integrated with HowNet to build a larger database.
- Noun Correlation Frame Extraction: Extracting correlation frames for nouns to represent modifier phrase order is another future direction.
- Further Research: The corpus is intended to support more statistical research, auto-tagging of semantic dependency structures, and applications in speech recognition, natural language understanding, machine translation, and information extraction.
Summary updated