Introduction
Search engines rely heavily on clickthrough logs to understand user intent and relevance. However, these logs are inherently noisy due to position bias—where users click top results regardless of relevance—and data sparseness, particularly in the "long tail" of rare queries. Traditional models often focus on "co-clicks" but ignore the valuable information in "skips"—results that were presented but not clicked. This paper proposes Collaborative Ranking (CoRank), a framework that addresses these issues by combining pairwise preference learning with matrix factorization.
Methodology
The CoRank framework unifies "learning to rank" with latent factor models to create a robust representation of query-URL relevance.
Pairwise Preferences
To mitigate position bias, the model shifts from absolute click counts to relative judgments. If a user clicks a lower-ranked URL $j$ while skipping a higher-ranked URL $k$, it implies a preference $j \succ k$. This approach captures the user's intent more accurately than treating clicks in isolation.
Matrix Factorization and Optimization
To handle data sparseness, CoRank maps queries and URLs into a shared latent factor space. This allows the model to infer relevance for rare queries by leveraging patterns from similar queries. The optimization uses the Bradley-Terry model to define preference likelihood, maintaining linear computational complexity relative to the number of preference pairs, ensuring scalability for production-level datasets.
Hybrid Model
The authors noted that CoRank and traditional random walk models (which focus on clicks) provide orthogonal information. By combining them into a linear hybrid model, the strengths of both "click" and "skip" signals are harnessed.
Experimental Evaluation
The model was tested on a massive dataset from a commercial search engine, comprising over 1.6 million queries and nearly 10 million URLs. Performance was measured by the accuracy of predicted pairwise preferences on a held-out test set.
Results
The experimental results demonstrate the superiority of the CoRank and Hybrid approaches over standard random walk models.
The Hybrid model achieved an accuracy of 0.94, a significant improvement over individual models. This confirms that collaborative ranking and click-based reinforcement are highly complementary.
Conclusion and Future Work
CoRank effectively mines the "long tail" of search logs by focusing on relative user preferences and latent factors. While it significantly improves ranking accuracy, future work will address the "cold start" problem for new queries and URLs, and explore the integration of content-based features and confidence weighting for different types of clicks.
Summary updated