Learning Hostname Preference to Enhance Search Relevance

Jingjing Wang, Changsung Kang, Yi Chang, Jiawei Han

Abstract

Search engines traditionally focus on query-to-URL relevance. This paper presents a framework for learning hostname preferences—the affinity between specific queries and domains—to improve search ranking. By integrating explicit features with latent factor modeling through a scalable, feature-aware matrix completion algorithm, the authors demonstrate significant improvements in Yahoo! Search relevance. The proposed Spark-based distributed implementation achieves high efficiency, processing billions of entries in minutes.

Introduction

The domain of a URL, or hostname, serves as a powerful signal for content type and user intent. For instance, a user searching for "historical facts" might prefer wikipedia.org, while "flight deals" suggests a preference for travel aggregators like expedia.com. Capturing these preferences requires modeling both explicit intent (keywords) and latent intent (behavioral patterns). However, the scale of search data—millions of queries and hostnames—poses a significant challenge for traditional matrix factorization techniques.

Methodology: Feature-Aware Matrix Completion

The authors propose a Feature-Aware Matrix Completion framework that combines global bias, explicit features (e.g., query keywords), and latent vectors. This hybrid approach allows the model to generalize across similar queries and hostnames while maintaining the precision of keyword-based signals.

To handle the massive scale of Yahoo! search logs, a block-wise parallel algorithm was developed on Apache Spark. A key innovation is the "fusion" of feature weight updates with latent vector updates within the Alternating Least Squares (ALS) framework. This optimization reduces communication overhead by a third, significantly accelerating convergence.

Convergence of train and test RMSE over iterations for various filtering parameters.

Experimental Results

The model was trained on a dataset of 1.3 billion click entries derived from 5.8 billion raw logs. Evaluation on an editorial dataset showed that incorporating learned hostname preferences as features in a production-grade ranking system yielded a 2% relative gain in Discounted Cumulative Gain (DCG). In the competitive field of search ranking, a 2% improvement from a single feature set is highly significant.

A critical finding was the impact of the update sequence. Updating feature weights ($w$) before latent factors led to superior local optima and faster convergence. As shown in the comparative analysis, the "w updates first" strategy consistently outperformed pure matrix factorization and alternative update orders in terms of test RMSE.

Comparison of RMSE across different update strategies and model ranks. The "w updates first" approach demonstrates the lowest test RMSE.

Efficiency and Scalability

The distributed algorithm demonstrated exceptional performance, completing the optimization of a million-by-million matrix in under 20 minutes on a 20-node cluster (at rank 50). This efficiency makes the framework viable for frequent retraining on fresh search data, ensuring that the learned preferences stay current with evolving web trends.

Conclusion

Learning hostname preferences provides a robust signal for search relevance by abstracting intent above the URL level. The proposed feature-aware matrix completion framework effectively balances explicit and latent signals. Its scalable implementation on Spark proves that complex, hybrid models can be trained on web-scale data with high efficiency, offering a significant boost to real-world search ranking performance.

Summary updated

Abstract

Hostnames such as en.wikipedia.org and www.amazon.com are strong indicators of the content they host. The relevant hostnames for a query can be a signature that captures the query intent. In this study, we learn the hostname preference of queries, which are further utilized to enhance search relevance. Implicit and explicit query intent are modeled simultaneously by a feature aware matrix completion framework. A block-wise parallel algorithm was developed on top of the Spark MLlib for fast optimization of feature aware matrix completion. The optimization completes within minutes at the scale of a million × million matrix, which enables efficient experimental studies at the web scale. Evaluation of the learned hostname preference is performed both intrinsically on test errors, and extrinsically on the impact on search ranking relevance. Experimental results demonstrate that capturing hostname preference can significantly boost the retrieval performance.
BibTeX
@inproceedings{wang2016learning,
  author = {Jingjing Wang and Changsung Kang and Yi Chang and Jiawei Han},
  title = {{Learning Hostname Preference to Enhance Search Relevance}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}
Figure