Introduction
The Knowledge Base Population (KBP) track at the 2009 Text Analysis Conference (TAC) presented significant challenges in extracting structured information from vast, unstructured datasets. This paper details the approach taken by the PRIS lab, focusing on two core tasks: Entity Linking (EL) and Slot Filling (SF). Their primary innovation lies in the use of query-specific automatic text summarization to perform cross-document coreference resolution, effectively filtering out noise that typically plagues full-document analysis.
Problem Statement
KBP involves two primary hurdles:
- Entity Linking (EL): Resolving ambiguous name mentions (e.g., "John Smith") to a specific entry in a large-scale Knowledge Base (KB) like Wikipedia.
- Slot Filling (SF): Automatically extracting attributes (e.g., birth date, headquarters) for specific entities from a diverse corpus including news, blogs, and forum posts.
The scale and linguistic variety of these sources make identifying relevant information difficult, as critical data points are often buried within long, noisy documents.
Methodology
Entity Linking (EL)
The PRIS lab implemented a multi-stage pipeline for Entity Linking. Initially, the Indri Retrieval Toolkit was used to fetch the top candidate documents from the KB. To combat the noise inherent in full-text comparison, the team generated query-specific summaries.
A simplified Hobbs algorithm was applied for intra-document pronoun resolution to ensure summaries remained contextually rich. The system then compared these summaries using two distinct similarity models: the Vector Space Model (VSM) and the Kullback-Leibler (KL) divergence model. A link was established only if entity types (identified via the Stanford Named Entity Recognizer) matched and similarity scores exceeded predefined thresholds.
Slot Filling (SF)
The Slot Filling system utilized a hybrid approach:
- Unsupervised Extraction: Relied on "trigger words" (e.g., "born in") to identify potential slot values based on proximity.
- Semi-supervised Learning: Employed Conditional Random Fields (CRFs) trained on 14 feature types, including POS tags and entity proximity.
The system prioritized unsupervised results, falling back to CRF-based predictions when necessary.
Key Results and Analysis
Entity Linking Performance
The initial submission to the TAC competition achieved a Micro-average of 0.3015. However, post-competition optimization of similarity thresholds led to a significant performance boost.
As shown in the refined results, the KL divergence model achieved a Micro-average of 0.6734 and a Macro-average of 0.7316. The researchers found that treating documents as probability distributions (KL divergence) provided a more robust signal for identity than simple vector overlap (VSM).
Slot Filling Performance
In the Slot Filling task, the system achieved an SF-value score of 0.462 for the top 5 results. The unsupervised trigger-word method proved more reliable than the CRF models, likely due to a domain mismatch between the KB-derived training data and the web-based test data.
Insights and Future Directions
The core insight of this work is that summarization acts as a powerful noise filter. By focusing on sentences directly relevant to the query and resolving internal pronouns, the system creates a cleaner "fingerprint" for entity matching.
However, limitations remain. Retrieval precision at the candidate generation stage is a bottleneck, and automatic summaries can still be redundant if source sentences are overly long. Future work will focus on more sophisticated NLP techniques to handle the informal linguistic styles found in web documents and improving the alignment between training and test domains for slot filling.
Summary updated