Abstract
This paper describes the ZJU_DCD_SF system developed for the TAC KBP 2015 Cold Start Slot Filling task. The system addresses the challenge of extracting entity attributes from unstructured text without extensive hand-labeled data. By integrating distant supervision with a Multi-Instance Multi-Label (MIML) learning framework, the approach robustly handles noisy training labels. Experimental results compare traditional lexical-syntactic features against word embedding-based features, highlighting the complexities of the slot-filling task.
Introduction
The paper tackles the challenge of Slot Filling within the context of Knowledge Base Population (KBP). Given an entity (such as a person or organization) and a massive corpus of unstructured text, the goal is to automatically extract specific attributes or "slot fillers" (e.g., a person's birthplace or an organization's founder). This pipeline requires solving several sub-problems: recognizing entity aliases, resolving coreferences, retrieving relevant documents, and accurately classifying relations between entities. Because high-quality, hand-annotated training data is prohibitively expensive to obtain, the authors focus on automated methods for data generation and robust classification.
Methodology
To build their slot-filling pipeline, the authors chain together several classical NLP techniques and machine learning models:
- Preprocessing & Candidate Retrieval: The system uses Stanford CoreNLP for tokenization, Named Entity Recognition (NER), coreference resolution, and dependency parsing. Candidate sentences are retrieved by expanding entity queries using aliases and searching the corpus via the Whoosh engine.
- Distant Supervision: To generate training data without manual labeling, the system assumes that if a knowledge base indicates a relation between two entities, any sentence containing both entities expresses that relation.
- Multi-Instance Multi-Label (MIML) Classification: Because distant supervision is inherently noisy, the authors utilize a MIML model. This model assumes that while an entity pair might have multiple valid relation labels, each specific text mention has exactly one latent (hidden) relation label.
As shown in the graphical model above, a two-layer hierarchical classifier aggregates mention-level latent variables into a final relation prediction for the entity pair, allowing the system to learn which mentions are most trustworthy.
Feature Engineering
The primary model relies heavily on Mintz-style features, including lexical features (words between entities, part-of-speech tags) and syntactic features (dependency paths). A secondary experimental model was developed using word2vec embeddings and position features to test the efficacy of early neural representations against structured linguistic features.
Key Results
The team submitted two runs to the TAC KBP 2015 Cold Start slot-filling track. The results, summarized in the table below, indicate that the model relying on classical lexical and syntactic features (ZJU_DCD_SF1) slightly outperformed the embedding-based model (ZJU_DCD_SF2).
While the absolute scores appear low, they are reflective of the extreme difficulty and strict evaluation metrics of the TAC KBP Cold Start tasks during that era. The results highlight that explicit syntactic structures remained highly competitive against early word embedding approaches for structured relation extraction.
Conclusion and Future Work
The study demonstrates the utility of combining distant supervision with latent variable models to scale relation extraction. However, the results also reveal the limitations of traditional feature engineering. The authors suggest that future work should explore neural network architectures, such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), to automatically capture semantic relationships and context without the need for manual feature design.
Summary updated