Event-Centric Query Expansion in Web Search

Yanan Zhang, Weijie Cui, Yangfan Zhang, Xiaoling Bai, Zhe Zhang, Jin Ma, Xiang Chen, Tianhua ZhouACL 2023

TL;DR

This paper introduces Event-Centric Query Expansion (EQE), a novel system designed to rapidly and accurately capture user intent for time-sensitive events in web search. EQE addresses the limitations of traditional query expansion methods by employing a four-stage pipeline: event collection, reformulation, semantic retrieval, and online ranking, leading to significant improvements in search performance.

What Problem Does This Paper Address?

Web search engines strive to provide users with the most relevant results, but understanding user intent, especially for rapidly evolving events, is a significant challenge. Traditional query expansion (QE) methods often rely on mining long-term search logs, which are too slow for time-sensitive events like breaking news. This leads to suboptimal search experiences when users are looking for the latest information. Existing approaches that use query-associated documents or entities also struggle in real-time scenarios. The paper aims to bridge this gap by developing a system that can quickly identify and leverage relevant events to expand queries, thereby improving the retrieval of event-related documents.

Methods

The EQE system is built upon a four-stage architecture:

  1. Event Collection: This stage focuses on identifying relevant events from news headlines. It uses a two-step filtering process:
  2. Coarse Filter: Employs rule-based filters (publication time, site type, etc.) and the LTP toolkit to extract event triggers and remove headlines with missing or multiple events.
  3. Fine Filter: Trains a RoBERTa-based event detection model on expert-annotated data to achieve a high accuracy (95%) in identifying event-related headlines.
  4. Event Reformulation: To address noise, irregular grammar, and low-frequency words in headlines, a generation model is used. This model incorporates two key techniques:
  5. Prompt Learning: Adaptive keyword templates are generated using KeyBERT to guide the sentence generation process, ensuring essential information is captured.
  6. Contrastive Learning: This technique is applied to the encoder's [CLS] token embeddings to combat representation space degradation, pulling similar headline representations closer and pushing dissimilar ones apart.
  7. Semantic Retrieval: A dual-tower semantic model, enhanced by a two-stage contrastive learning approach, is used to retrieve relevant events for a given query. This method addresses the issue of uninformative negative samples by first training a retrieval model with positive samples and shuffled negatives, then retraining it with hard negative samples identified from distribution statistics.
  8. Online Ranking: A Gradient Boosting Decision Tree (GBDT) model is employed to select the optimal event expansion from the retrieved candidates. It utilizes features from the query (domain classification, entity recognition), the event (found time, popularity), and their interaction (semantic similarity, BM25 score).

Datasets

  • News Headlines: A dataset of approximately 50 million news article headlines collected over six months was used for event collection.
  • Title2EventPhrase: A newly introduced dataset containing over 41,000 Chinese news headlines and their corresponding event phrases, annotated by human experts. This dataset was used for training and evaluating the event reformulation model.
  • User Click-Logs: Used for constructing training pairs of <headline, event> for the event reformulation stage.
  • Online A/B Testing Data: Collected from Tencent QQ Browser Search, involving millions of users over 30 days.

Key Insights and Ideas

  • Event-centricity is Key for Timeliness: The core idea is that focusing on specific, real-world events is crucial for understanding and satisfying user intent in time-sensitive search scenarios, especially for news.
  • Generation Over Extraction for Reformulation: Using a generative model to reformulate headlines is more effective than extractive methods for creating concise and high-quality event descriptions.
  • Contrastive Learning for Robust Embeddings: Applying contrastive learning, particularly in both event reformulation and semantic retrieval, significantly improves the quality and discriminative power of learned representations, mitigating issues like representation space degradation.
  • Two-Stage Training for Better Retrieval: The two-stage contrastive learning approach for semantic retrieval, which strategically uses hard negative samples, is vital for improving the accuracy of identifying relevant events.
  • Hybrid Approach for Online Ranking: Combining query-side, event-side, and interaction features in a GBDT model provides a robust way to select the best event expansion in real-time.

Key Results

  • Offline Evaluation: EQE significantly surpasses the baseline in Recall@K metrics, indicating better retrieval performance.
  • Online A/B Testing: EQE demonstrated strong performance on key business metrics, achieving improvements of 6.44% on CTR, 6.23% on PCTR, and 5.03% on UCTR compared to the baseline.
  • Event Collection Coverage: EQE discovers events over 10% earlier than other systems at the 5-minute mark after an event occurs.
  • Event Reformulation: The proposed components (contrastive learning and prompt guidance) significantly improved generation performance as measured by ROUGE, BLEU, and BERTScore metrics.
  • Semantic Retrieval: The two-stage training scheme with hard negative samples led to substantial improvements in Recall@10, MRR@10, and AUC compared to baseline semantic models.
  • System Efficiency: The EQE system adds only about 10 ms to the search latency, demonstrating its efficiency for real-time deployment.

Limitations and Future Work

  • Scope of Events: The paper focuses on events primarily from news websites and does not cover all types of user intents, particularly those requiring explicit knowledge beyond current events.
  • Dataset Bias: While efforts were made to create a diverse dataset (Title2EventPhrase), the inherent biases of news reporting and platform-specific content might still influence results.
  • Future Work: The authors suggest that further research could explore more sophisticated event representation methods and investigate how to integrate EQE with other intent understanding modules to cover a broader range of user queries. The system has been deployed in Tencent QQ Browser Search, serving millions of users, which provides a strong foundation for future iterations and improvements.

Summary updated

BibTeX
@inproceedings{zhang2023event,
  author = {Yanan Zhang and Weijie Cui and Yangfan Zhang and Xiaoling Bai and Zhe Zhang and Jin Ma and Xiang Chen and Tianhua Zhou},
  title = {{Event-Centric Query Expansion in Web Search}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure