ESA: Entity Summarization with Attention

Dongjun Wei, Yaxin Liu, Fuqing Zhu, Liangjun Zang, Wei Zhou, Jizhong Han, Songlin Hu

Abstract

Entity Summarization (ES) aims to extract a concise set of informative facts from large-scale Knowledge Graphs (KGs). Traditional methods rely on heuristic graph-based metrics which often fail to capture human preferences. We present ESA (Entity Summarization with Attention), a deep learning approach that utilizes a Bidirectional Long Short-Term Memory (BiLSTM) network combined with a supervised attention mechanism. By training the attention layer directly on human-curated summaries, ESA learns to predict the importance of RDF triples. Experimental results on the ESBM benchmark demonstrate that ESA significantly outperforms state-of-the-art traditional methods in both F-measure and Mean Average Precision (MAP).

Introduction

Knowledge Graphs (KGs) such as DBpedia and YAGO have become foundational for information retrieval and semantic search. However, the sheer volume of data—where a single entity can be associated with thousands of RDF triples—necessitates effective summarization techniques. The goal of Entity Summarization is to select the top-$k$ triples that best represent an entity to a human user.

Historically, this task was addressed through hand-crafted logic, including PageRank-style centrality, diversity-based clustering, or complex rule-based ranking. These methods often struggle to generalize across different domains or capture the nuances of human interest. ESA addresses this gap by framing summarization as a supervised learning task, leveraging attention mechanisms to mimic human editorial judgment.

Methods

The ESA architecture is designed as an encoder-decoder-style neural network that processes sequences of RDF triples.

Knowledge Representation

To process triples numerically, ESA employs a hybrid embedding strategy. Predicates are mapped using standard word embeddings to capture linguistic meaning. Objects are represented using TransE, a graph embedding technique that preserves structural relationships within the KG. This combination provides a richer representation than linguistic or structural features alone.

BiLSTM Encoder and Supervised Attention

The core of the model is a BiLSTM network that processes the sequence of triples to capture contextual dependencies. While triples are technically unordered, the BiLSTM helps the model understand how facts co-occur and relate within the context of a specific entity.

The ESA model architecture, featuring hybrid embeddings, a BiLSTM encoder, and a supervised attention layer.

The defining innovation of ESA is its Supervised Attention mechanism. Unlike standard attention that learns weights implicitly, ESA is trained using "gold" attention vectors. These vectors are derived from the frequency with which specific triples were selected by humans in existing benchmarks. The model is explicitly optimized to minimize the divergence between its predicted attention weights and these human-derived ground truths.

Results

The model was evaluated using the ESBM (Entity Summarization BenchMark) v1.1, which includes 125 entities from DBpedia and 50 from LinkedMDB. ESA was compared against several state-of-the-art traditional models, including RELIN, FACES, and CD.

F-measure results for ESA compared to baseline methods across DBpedia and LinkedMDB datasets.

As shown in the results, ESA consistently outperforms baseline models. On the DBpedia dataset ($k=5$), ESA achieved an F-measure of 0.310, surpassing the previous best (CD) at 0.287. Similar gains were observed in Mean Average Precision (MAP), particularly on the LinkedMDB dataset, where ESA reached 0.369 ($k=5$) compared to 0.341 for the next best method. Overall, ESA demonstrated a 7.96% improvement in F-measure for top-5 summaries across the combined datasets.

Conclusion

ESA demonstrates the effectiveness of supervised attention for the task of entity summarization. By directly learning from human preferences and combining linguistic and structural embeddings, the model provides more relevant and informative summaries than traditional heuristic methods. Future work will focus on scaling the training data and exploring permutation-invariant architectures like Graph Neural Networks to better handle the unordered nature of RDF triples.

Summary updated

Abstract

Entity summarization aims at creating brief but informative descriptions of entities from knowledge graphs. While previous work mostly focused on traditional techniques such as clustering algorithms and graph models, we ask how to apply deep learning methods into this task. In this paper we propose ESA, a neural network with supervised attention mechanisms for entity summarization. Specifically, we calculate attention weights for facts in each entity, and rank facts to generate reliable summaries. We explore techniques to solve difficult learning problems presented by the ESA, and demonstrate the effectiveness of our model in comparison with the state-of-the-art methods. Experimental results show that our model improves the quality of the entity summaries in both F-measure and MAP.
BibTeX
@inproceedings{wei2019esa,
  author = {Dongjun Wei and Yaxin Liu and Fuqing Zhu and Liangjun Zang and Wei Zhou and Jizhong Han and Songlin Hu},
  title = {{ESA: Entity Summarization with Attention}},
  booktitle = {EYRE @ CIKM},
  year = {2019},
  url = {https://openreview.net/forum?id=Y728eYBOhw},
}
Figure