Towards Better Text Understanding and Retrieval through Kernel Entity Salience Modeling

Chenyan Xiong, Zhengzhong Liu, Jamie Callan, Tie-Yan Liu

This paper, presented at SIGIR 2018, introduces the Kernel Entity Salience Model (KESM), a neural approach designed to move information retrieval (IR) beyond simple word-matching and frequency counts toward a deeper "understanding" of what a document is actually about.

TL;DR

KESM improves document retrieval by accurately identifying "salient" entities—the key people, places, or concepts central to a text—rather than just counting how often they appear. By using neural kernels to model interactions between entities and their surrounding context, the model significantly boosts performance in both entity importance estimation and ad hoc web search.

What Problem Does This Paper Address?

In traditional Information Retrieval, we often use term frequency (TF) as a proxy for importance: if a word appears many times, it must be important. However, frequency does not equal salience. For example, a person’s homepage might mention their name only once, while a Wikipedia article might mention a secondary entity frequently without it being the "main point."

While word-based retrieval is mature, "entity-oriented" search—which represents documents as collections of linked entities (like those in a Knowledge Graph)—has struggled with noisy entity annotations and a lack of sophisticated ways to weight those entities. The authors aim to bridge this gap by creating a model that learns to estimate entity importance (salience) end-to-end.

Methods

The KESM architecture consists of two primary components:

  1. Knowledge Enriched Embedding (KEE): Instead of using static word vectors, KESM represents entities by combining their learned embeddings with their descriptions (e.g., the first paragraph of their Wikipedia page). A Convolutional Neural Network (CNN) processes these descriptions to create a "knowledge-enriched" vector that captures the entity's broader semantic meaning.
  2. Kernel Interaction Model (KIM): This is the core innovation. To determine if an entity $e$ is salient in document $d$, KIM looks at how $e$ interacts with every other word and entity in that document. It uses a series of RBF (Radial Basis Function) kernels to "pool" similarities. Essentially, these kernels count how many words or entities in the document fall into specific "neighborhoods" of the target entity in the embedding space. This allows the model to capture "votes" of importance from the surrounding context.

The model is trained using a pairwise ranking loss, ensuring that salient entities (those identified in summaries or titles) are ranked higher than non-salient ones.

Datasets

The authors evaluated KESM on two distinct tasks:

  • Entity Salience Estimation:
    • New York Times (NYT): A corpus of ~526k news articles. Entities mentioned in the expert-written summary are labeled as "salient."
    • Semantic Scholar: ~800k scientific abstracts. Entities appearing in the paper's title are considered "salient."
  • Ad Hoc Search:
    • TREC Web Track (ClueWeb09 & ClueWeb12): Standard benchmarks for web search, containing 300 queries and millions of web pages.

Key Insights and Ideas

  • Frequency is a Crutch: The authors demonstrate that frequency-based models are heavily biased toward "head" (popular) entities. KESM, by contrast, is much better at identifying the salience of "tail" (rare) entities because it relies on semantic interactions rather than just raw counts.
  • Robustness on Short Text: Frequency signals are notoriously weak in short documents (like abstracts). KESM maintains high accuracy on short texts because its kernel-based interaction modeling provides a richer signal than a simple count.
  • Transferability: A major "aha" moment is that KESM can be pre-trained on the entity salience task (using news data) and then used as a feature extractor for general web search. The "text understanding" learned from identifying salient entities in news translates directly into better document ranking for search engines.

Key Results

  • Salience Task: KESM outperformed the state-of-the-art feature-based model (LeToR) by 7.5% to 18% in Precision@1 across the NYT and Semantic Scholar datasets.
  • Search Task: When used as a ranking feature on the TREC Web Track, KESM improved NDCG@20 by up to 13.5% over standard word-based baselines. It proved particularly effective when analyzing the "body" of documents, where deeper text understanding is most required.
  • Efficiency: Despite being a neural model, KESM is faster to train and deploy than traditional feature-based models that require expensive linguistic preprocessing like dependency parsing or coreference resolution.

Limitations and Future Work

The authors note that KESM currently focuses on modeling the salience of entities within the document. However, it treats all entities in the query as equally important. In reality, some query terms are more critical than others.

Future work could involve applying similar kernel-based salience modeling to the query side to better understand search intent. Additionally, the model still relies on the quality of the initial entity linking (e.g., TagMe), meaning errors in the linking phase can propagate through the system.

Summary updated

Abstract

This paper presents a Kernel Entity Salience Model (KESM) that improves text understanding and retrieval by better estimating entity salience (importance) in documents. KESM represents entities by knowledge enriched distributed representations, models the interactions between entities and words by kernels, and combines the kernel scores to estimate entity salience. The whole model is learned end-to-end using entity salience labels. The salience model also improves ad hoc search accuracy, providing effective ranking features by modeling the salience of query entities in candidate documents. Our experiments on two entity salience corpora and two TREC ad hoc search datasets demonstrate the effectiveness of KESM over frequency-based and feature-based methods. We also provide examples showing how KESM conveys its text understanding ability learned from entity salience to search.
BibTeX
@inproceedings{xiong2018towards,
  author = {Chenyan Xiong and Zhengzhong Liu and Jamie Callan and Tie-Yan Liu},
  title = {{Towards Better Text Understanding and Retrieval through Kernel Entity Salience Modeling}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2018},
  url = {https://openreview.net/forum?id=HyE5tSW_ZH},
}

Loading related content...

Figure