Open Relation Extraction (ORE) is a critical task in Natural Language Processing, focused on extracting structured facts—typically in the form of (Subject, Relation, Object) triples—from unstructured text without a pre-defined schema. While ORE has seen significant success in English, the Chinese language presents unique challenges that render standard English-centric approaches less effective. This paper introduces ZORE, a syntax-based system specifically engineered to handle the linguistic nuances of Chinese.
The Challenges of Chinese ORE
Extracting relations from Chinese text is complicated by three primary factors:
- Flexible Word Order: Unlike English, where arguments often follow a predictable linear order relative to the predicate, Chinese arguments can appear on either side, making surface-pattern matching unreliable.
- Light Verb Constructions (LVCs): Chinese frequently employs "empty" verbs (e.g., "make" in "make a contribution"). Extracting only the verb leads to uninformative relations; the system must identify the full predicate phrase.
- Parsing Vulnerability: Automatic syntactic and semantic parsers for Chinese are generally more error-prone than their English counterparts. Downstream extraction systems often suffer from "garbage in, garbage out" due to these initial parsing inaccuracies.
System Architecture and Methodology
ZORE addresses these hurdles through a three-stage pipeline designed to reinforce syntactic analysis with semantic knowledge.
1. Candidate Extraction
The system begins by processing raw text through segmentation, POS tagging, and dependency parsing. To solve the LVC problem, ZORE employs a statistical method to detect light verbs, ensuring that the extracted predicate captures the true action of the sentence (e.g., identifying "graduated from" as a single unit).
2. Double Propagation Algorithm
The core innovation of ZORE is the "double propagation" algorithm. Rather than a linear extraction process, ZORE uses an iterative loop where relations and semantic patterns are extracted simultaneously to reinforce one another:
- Initial Tagging: The system identifies unambiguous ("monosemic") arguments using the Extended Cilin taxonomy to establish baseline semantic patterns.
- Iterative Matching: These patterns are then used to resolve ambiguous or unknown words through perfect and partial matching.
- Error Correction: This two-way exchange allows high-confidence semantic patterns to effectively "correct" errors introduced during the initial syntactic parsing phase.
3. Filtering and Grouping
Finally, the system clusters extracted patterns into "synsets" (groups of synonymous patterns) and utilizes a logistic regression classifier to assign confidence scores, filtering out low-quality extractions.
Evaluation and Key Results
ZORE was evaluated against manually annotated datasets (Wiki-500 and Sina-500) and demonstrated superior performance compared to traditional ORE baselines.
On the Wikipedia dataset, ZORE achieved a precision of 0.77 at a recall of 0.3, outperforming the baseline by 0.11. Beyond precision, the system's scalability was proven by running it on over 10 million sentences, resulting in a released dataset of 5 million relations and 344,000 semantic patterns. Human evaluation of the top 100 pattern synsets revealed an average precision of 92%, validating the effectiveness of the double propagation method in grouping synonymous expressions.
Conclusion
ZORE demonstrates that for Chinese ORE, the integration of deep syntax and semantic taxonomies is essential. By treating extraction as a bootstrapping problem—where patterns and relations inform each other—the system overcomes the inherent limitations of automatic parsing. While parsing errors remain a challenge, accounting for over half of incorrect extractions, ZORE provides a robust framework and a massive resource for the Chinese NLP community.
Summary updated