Abstract
This paper presents a Kernel Entity Salience Model (KESM) that improves text understanding and retrieval by better estimating entity salience (importance) in documents. KESM represents entities by knowledge enriched distributed representations, models the interactions between entities and words by kernels, and combines the kernel scores to estimate entity salience. The whole model is learned end-to-end using entity salience labels. The salience model also improves ad hoc search accuracy, providing effective ranking features by modeling the salience of query entities in candidate documents. Our experiments on two entity salience corpora and two TREC ad hoc search datasets demonstrate the effectiveness of KESM over frequency-based and feature-based methods. We also provide examples showing how KESM conveys its text understanding ability learned from entity salience to search.
Introduction
In traditional Information Retrieval (IR), term frequency (TF) has long served as a proxy for importance. The underlying assumption is simple: if a word appears many times, it must be central to the document's meaning. However, frequency does not always equate to salience. For instance, a person’s homepage might mention their name only once, while a Wikipedia article might mention a secondary entity frequently without it being the primary subject.
While entity-oriented search—representing documents as collections of linked entities from a Knowledge Graph—offers a path toward deeper text understanding, it has historically struggled with noisy annotations and simplistic weighting schemes. This paper introduces the Kernel Entity Salience Model (KESM), a neural approach designed to accurately identify salient entities by modeling semantic interactions rather than relying on raw counts.
Methodology
The KESM architecture consists of two primary components designed to move beyond surface-level matching:
1. Knowledge Enriched Embedding (KEE)
Instead of using static word vectors, KESM represents entities by combining their learned embeddings with their descriptions (e.g., the introductory paragraph of their Wikipedia page). A Convolutional Neural Network (CNN) processes these descriptions to create a "knowledge-enriched" vector. This ensures that the representation captures the entity's broader semantic context, making it more robust than a simple ID-based embedding.
2. Kernel Interaction Model (KIM)
To determine if an entity $e$ is salient in document $d$, KIM analyzes how $e$ interacts with every other word and entity in that document. It utilizes a series of Radial Basis Function (RBF) kernels to pool similarities. These kernels essentially count how many words or entities in the document fall into specific semantic "neighborhoods" of the target entity. This allows the model to capture "votes" of importance from the surrounding context, effectively learning which types of interactions signal salience.
The model is trained end-to-end using a pairwise ranking loss, ensuring that salient entities (those identified in summaries or titles) are ranked higher than non-salient ones.
Experimental Evaluation
The authors evaluated KESM on two distinct tasks: entity salience estimation and ad hoc search.
- Entity Salience Estimation: Tested on the New York Times (NYT) corpus (~526k articles) and Semantic Scholar (~800k abstracts). Salience was determined by presence in expert summaries or paper titles, respectively.
- Ad Hoc Search: Evaluated on the TREC Web Track (ClueWeb09 & ClueWeb12) benchmarks, containing 300 queries and millions of web pages.
Key Results
KESM demonstrated significant improvements over state-of-the-art feature-based models (LeToR), with gains of 7.5% to 18% in Precision@1. In the search task, KESM improved NDCG@20 by up to 13.5% over standard word-based baselines, proving particularly effective when analyzing document bodies where deep understanding is required.
Discussion: Why KESM Works
A critical insight from the results is that frequency-based models are heavily biased toward "head" (popular) entities. As shown in the frequency bin analysis, KESM is significantly better at identifying the salience of "tail" (rare) entities because it relies on semantic interactions. Furthermore, KESM maintains high accuracy on short texts (like scientific abstracts) where frequency signals are notoriously weak.
Perhaps most importantly, the model exhibits high transferability. The "text understanding" learned from identifying salient entities in news data translates directly into better document ranking for general web search, allowing KESM to serve as a powerful feature extractor for IR systems.
Conclusion
KESM represents a step forward in bridging the gap between keyword matching and semantic understanding. By modeling entity salience through kernel-based interactions, it provides a robust, end-to-end framework for both entity importance estimation and document retrieval. Future work may explore applying similar kernel-based modeling to the query side to better capture search intent and address the current limitation of treating all query entities as equally important.
Summary updated