Abstract
This paper presents a word-entity duet framework for utilizing knowledge bases in ad-hoc retrieval. In this work, the query and documents are modeled by word-based representations and entity-based representations. Ranking features are generated by the interactions between the two representations, incorporating information from the word space, the entity space, and the cross-space connections through the knowledge graph. To handle the uncertainties from the automatically constructed entity representations, an attention-based ranking model AttR-Duet is developed. With back-propagation from ranking labels, the model learns simultaneously how to demote noisy entities and how to rank documents with the word-entity duet. Evaluation results on TREC Web Track ad-hoc task demonstrate that all of the four-way interactions in the duet are useful, the attention mechanism successfully steers the model away from noisy entities, and together they significantly outperform both word-based and entity-based learning to rank systems.
Introduction
Traditional information retrieval (IR) is primarily "word-centric," relying on keyword matching between queries and documents. While knowledge bases (KBs) like Freebase or DBpedia offer rich semantic information, they are often relegated to secondary roles like query expansion. This paper identifies two major hurdles in better utilizing KBs: the integration gap, where cross-space interactions (e.g., query entities vs. document words) are ignored, and annotation noise, where automatic entity linking often maps text to incorrect KB entries.
To address these, the authors introduce the Word-Entity Duet framework. This approach treats entities as first-class citizens alongside words, modeling a four-way interaction that captures semantic evidence across both spaces. By incorporating a novel attention mechanism to filter out noisy annotations, the system achieves significant performance gains over state-of-the-art retrieval models.
The Word-Entity Duet Framework
The core of the proposal is representing both queries and documents in two distinct spaces: the word space (Bag-of-Words) and the entity space (Bag-of-Entities). Instead of simple word-to-word matching, the duet calculates features across four channels:
- Qw-Dw (Word-Word): Standard IR features like BM25 and Language Models.
- Qe-Dw (Query Entity-Doc Word): Matches the names and descriptions of query entities against document text.
- Qw-De (Query Word-Doc Entity): Matches query words against the names and descriptions of entities found in the document.
- Qe-De (Entity-Entity): Uses knowledge graph embeddings (TransE) and "soft matching" to find semantic similarities between entities that do not share the same surface form (e.g., "Obama" and "White House").
AttR-Duet: Handling Annotation Noise
To manage the inherent noise in automatic entity linking, the authors developed AttR-Duet, a ranking model utilizing a shallow Convolutional Neural Network (CNN). The model's standout feature is an attention mechanism that weights query entities based on two factors:
- Ambiguity: Features like the entropy of the surface form (how many different entities a word could represent).
- Closeness: The cosine similarity between the entity's embedding and the query's word embeddings.
This allows the model to "demote" entities suspected of being incorrectly linked or irrelevant to the user's search intent, acting as a critical filter for the ranking process.
Experimental Results
The model was evaluated on the ClueWeb09-B and ClueWeb12-B13 benchmarks from the TREC Web Track, using Freebase for the knowledge graph and TagMe for entity annotation.
Key Findings
- Significant Performance Gains: AttR-Duet outperformed standard learning-to-rank baselines (like RankSVM) and previous entity-based methods (like ESR) by 14% to 31% across metrics such as NDCG@20.
- The Importance of Soft Matching: The Qe-De component demonstrated that identifying "neighboring" entities in a knowledge graph provides a powerful signal that overcomes the vocabulary mismatch problem.
- Robustness to Noise: On the ClueWeb12 dataset, where entity linking was particularly noisy, the attention mechanism was responsible for a massive 14% performance gain. Without it, the model was easily distracted by irrelevant annotations.
Conclusion
The Word-Entity Duet framework demonstrates that treating entities as peers to words—rather than mere add-ons—captures essential cross-space evidence. While the current model uses a shallow CNN and focuses on query-side attention, it establishes a robust foundation for future work, such as applying attention to document-side entities or utilizing deeper neural architectures to further refine the ranking signal.
Summary updated