An Introduction to BLCU Personal Attributes Extraction System

Dong Yu, Cheng Yu, Qin Qu, Gongbo Tang, Chunhua Liu, Yue Tian, Jing Yi2014 · DOI

TL;DR

This paper introduces the BLCU-PAE system for extracting personal attributes from Chinese text, a crucial task for information extraction. The system employs a multi-stage approach combining CRF models, regular expressions, and novel filtering techniques based on protagonist dependency and attribute keywords to improve accuracy.

What Problem Does This Paper Address?

Extracting specific personal attributes (like date of birth, family relationships, education, etc.) from unstructured Chinese text is a challenging task. Existing methods often struggle with accuracy, particularly in distinguishing attributes belonging to the main subject of the text from those of other individuals mentioned. This paper aims to improve the precision of personal attribute extraction by developing a system that can effectively handle these nuances.

Methods

The BLCU-PAE system utilizes a four-stage pipeline:

  1. Pre-process: Documents are segmented into sentences, and NLP techniques like word segmentation, POS tagging, Named Entity Recognition (NER), and dependency parsing are applied.
  2. First Step Extraction: This stage employs two main approaches:
  3. CRF Extractor: Used for attributes where context features are distinct and sufficient labeled data exists.
  4. Regular Expression Extractor: Employed for attributes with similar contexts (making CRF difficult) or where training data is sparse (e.g., "Religion").
  5. Results Refine: This is a key innovation. Two methods are used to filter out incorrect extractions:
  6. Protagonist Dependency Filter: Leverages dependency parsing to ensure extracted attribute values are linked to the main protagonist of the document, not other individuals.
  7. Attribute Keywords Filter: Uses high-frequency context words associated with specific attributes to identify and discard erroneous extractions.
  8. Post-process: Extracted values are combined, and their positions in the original document are determined to generate a final XML output.

The system also categorizes the 25 attributes into groups, assigning specific extraction pipelines to each category, rather than building a unique model for every attribute.

Datasets

The system was trained and evaluated using:

  • Training Data: Collected from approximately 40,000 biography pages from BaiduBaike and 6,000 pages from Wikipedia. This data was carefully selected and annotated.
  • Evaluation Data: The paper mentions a test set with 90 persons and 233 documents.

Key Insights and Ideas

  • Protagonist-centric extraction is crucial: Simply extracting attribute values isn't enough; ensuring they belong to the main subject of the text is vital for accurate personal attribute extraction. Dependency parsing is a powerful tool for this.
  • Hybrid approaches are effective: Combining supervised learning (CRF) with rule-based methods (regular expressions) and knowledge bases (country-state/province, family, religion databases) offers a robust solution.
  • Filtering is as important as extraction: The proposed protagonist dependency and keyword-based filters significantly improve precision by addressing common error types.
  • Attribute categorization simplifies system design: Grouping attributes and assigning tailored pipelines reduces complexity and computational cost.

Key Results

The paper reports evaluation results using metrics from the TAC KBP slot filling task. While specific numbers are provided in Table 5, the general findings are:

  • The system achieved a single score of 0.4286 (lenient) and 0.4113 (strict), and a list score of 0.1888 (lenient) and 0.1739 (strict).
  • The overall SF value was 0.3087 (lenient) and 0.2926 (strict).
  • The system's performance on single-value attributes was significantly better than on multi-value attributes, indicating that list-value extraction remains more challenging.
  • There is a notable gap between the BLCU-PAE system and the best performing system in the bakeoff.

Limitations and Future Work

  • Contextual understanding: The authors acknowledge that their system primarily operates at the sentence level. Future work could involve better utilization of context across sentences and at the document level, potentially through anaphora resolution and entity linking.
  • Data annotation challenges: Supervised learning heavily relies on annotated data, which is costly and time-consuming to create.
  • Bootstrapping and Active Learning: The paper suggests that bootstrapping methods for mining more samples and active learning frameworks could be more effective for knowledge coverage.
  • Improving recall for keyword filters: While keyword filters enhance accuracy, they can negatively impact recall. Balancing this trade-off is an area for improvement.

Summary updated

Abstract

We describe our methods for share task of personal attributes extraction.We divide all 25 attributes into several categories and propose 4 kinds of pipelines to carry out value extraction.There are two stages in the process.The first stage uses CRF model or regular expression based extractor to produce initial answers.In the second stage, we propose two methods to filter out mistake answers: protagonist dependency relationship based filter and attribute keywords based filter.
BibTeX
@misc{yu2014introduction,
  author = {Dong Yu and Cheng Yu and Qin Qu and Gongbo Tang and Chunhua Liu and Yue Tian and Jing Yi},
  title = {{An Introduction to BLCU Personal Attributes Extraction System}},
  year = {2014},
  doi = {10.3115/v1/w14-6819},
}
Figure