ZORE: A Syntax-based System for Chinese Open Relation Extraction

Likun Qiu, Yue ZhangEMNLP 2014 · DOI

Open Relation Extraction (ORE) is a critical task in Natural Language Processing, focused on extracting structured facts—typically in the form of (Subject, Relation, Object) triples—from unstructured text without a pre-defined schema. While ORE has seen significant success in English, the Chinese language presents unique challenges that render standard English-centric approaches less effective. This paper introduces ZORE, a syntax-based system specifically engineered to handle the linguistic nuances of Chinese.

The Challenges of Chinese ORE

Extracting relations from Chinese text is complicated by three primary factors:

  1. Flexible Word Order: Unlike English, where arguments often follow a predictable linear order relative to the predicate, Chinese arguments can appear on either side, making surface-pattern matching unreliable.
  2. Light Verb Constructions (LVCs): Chinese frequently employs "empty" verbs (e.g., "make" in "make a contribution"). Extracting only the verb leads to uninformative relations; the system must identify the full predicate phrase.
  3. Parsing Vulnerability: Automatic syntactic and semantic parsers for Chinese are generally more error-prone than their English counterparts. Downstream extraction systems often suffer from "garbage in, garbage out" due to these initial parsing inaccuracies.

System Architecture and Methodology

ZORE addresses these hurdles through a three-stage pipeline designed to reinforce syntactic analysis with semantic knowledge.

The ZORE system architecture, illustrating the pipeline from candidate extraction to final output.

1. Candidate Extraction

The system begins by processing raw text through segmentation, POS tagging, and dependency parsing. To solve the LVC problem, ZORE employs a statistical method to detect light verbs, ensuring that the extracted predicate captures the true action of the sentence (e.g., identifying "graduated from" as a single unit).

2. Double Propagation Algorithm

The core innovation of ZORE is the "double propagation" algorithm. Rather than a linear extraction process, ZORE uses an iterative loop where relations and semantic patterns are extracted simultaneously to reinforce one another:

  • Initial Tagging: The system identifies unambiguous ("monosemic") arguments using the Extended Cilin taxonomy to establish baseline semantic patterns.
  • Iterative Matching: These patterns are then used to resolve ambiguous or unknown words through perfect and partial matching.
  • Error Correction: This two-way exchange allows high-confidence semantic patterns to effectively "correct" errors introduced during the initial syntactic parsing phase.

3. Filtering and Grouping

Finally, the system clusters extracted patterns into "synsets" (groups of synonymous patterns) and utilizes a logistic regression classifier to assign confidence scores, filtering out low-quality extractions.

Evaluation and Key Results

ZORE was evaluated against manually annotated datasets (Wiki-500 and Sina-500) and demonstrated superior performance compared to traditional ORE baselines.

Precision-Recall curve comparing ZORE to a baseline system, showing significant improvements in precision across various recall levels.

On the Wikipedia dataset, ZORE achieved a precision of 0.77 at a recall of 0.3, outperforming the baseline by 0.11. Beyond precision, the system's scalability was proven by running it on over 10 million sentences, resulting in a released dataset of 5 million relations and 344,000 semantic patterns. Human evaluation of the top 100 pattern synsets revealed an average precision of 92%, validating the effectiveness of the double propagation method in grouping synonymous expressions.

Conclusion

ZORE demonstrates that for Chinese ORE, the integration of deep syntax and semantic taxonomies is essential. By treating extraction as a bootstrapping problem—where patterns and relations inform each other—the system overcomes the inherent limitations of automatic parsing. While parsing errors remain a challenge, accounting for over half of incorrect extractions, ZORE provides a robust framework and a massive resource for the Chinese NLP community.

Summary updated

BibTeX
@inproceedings{qiu2014zore,
  author = {Likun Qiu and Yue Zhang},
  title = {{ZORE: A Syntax-based System for Chinese Open Relation Extraction}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2014},
  doi = {10.3115/v1/d14-1201},
}
Figure