Introduction
Building comprehensive knowledge graphs requires the accurate extraction of entity attributes—specific facts like birthplace or occupation—from unstructured text. This task, known as Entity Attribute Extraction, faces significant challenges including data scarcity, linguistic variety, and extreme class imbalance. Traditional models often struggle with the "long tail" of rare attributes. Researchers from the National University of Defense Technology have proposed a Siamese LSTM architecture to address these issues by learning highly discriminative sentence embeddings through pair-wise training.
Methodology
The core of the proposed approach is a Siamese network designed to recognize semantic similarities between different sentences. Unlike standard classifiers that process single instances, this model evaluates pairs of sentences to determine if they express the same attribute.
Model Architecture
The architecture consists of two identical LSTM sub-networks that share weights. The pipeline begins with converting sentences into 50-dimensional GloVe word embeddings. To provide context regarding the entities involved, "marker embeddings" are added to identify the positions of the two entities being linked.
As shown in the architecture diagram, the tokens and entity markers pass through word and position embedding layers before entering the LSTM layers. These layers produce an "attribute vector" which represents the semantic relationship within the sentence. Finally, a Softmax classifier uses these refined vectors to predict the final attribute type.
Siamese Training and Contrastive Loss
During training, the model is fed pairs of sentences. "Similar pairs" express the same attribute, while "dissimilar pairs" express different ones. A contrastive loss function is employed to minimize the distance between embeddings of similar pairs and maximize the distance for dissimilar ones.
This pair-wise training effectively augments the dataset, scaling the number of training samples from $O(N)$ to $O(N^2)$. This is particularly beneficial for rare attributes, as the model can be trained on many more combinations of examples. By forcing the model to compare sentences, it becomes robust to variations in syntax and vocabulary, learning that the underlying relationship is what matters most.
Experimental Results
The model was evaluated on a large-scale dataset of 193,783 sentences from Wikipedia, aligned with 41 distinct Wikidata attribute labels. The Siamese model consistently outperformed baseline CNN and standard LSTM models.
Key findings include:
- Performance on Rare Classes: For attributes with fewer than 1,000 samples, the Siamese model showed F1-score improvements of over 10% compared to baselines. For example, in the "cause of death" category, it significantly outperformed standard models.
- Precision-Recall: The Siamese-8-1-1 model maintained high precision even as recall increased, demonstrating superior stability.
- Clustering Quality: Visual analysis confirmed that the Siamese approach creates tighter, more distinct attribute clusters compared to traditional methods, which often produce overlapping groups.
Conclusion
By leveraging Siamese networks for attribute extraction, this research demonstrates a powerful method for overcoming data scarcity and class imbalance. The ability to learn discriminative embeddings through sentence comparison allows for more accurate and robust information extraction, paving the way for more complete and reliable knowledge graphs. Future work will explore extending this method to other entity domains beyond person entities and exploring automated knowledge fusion.
Summary updated