Abstract
Position bias is a critical problem in information retrieval when dealing with implicit yet biased user feedback data. Unbiased ranking methods typically rely on causality models and debias the user feedback through inverse propensity weighting. While practical, these methods still suffer from two major problems. First, when inferring a user click, the impact of the contextual information, such as documents that have been examined, is often ignored. Second, only the position bias is considered but other issues resulted from user browsing behaviors are overlooked. In this paper, we propose an end-to-end Deep Recurrent Survival Ranking (DRSR), a unified framework to jointly model user's various behaviors, to (i) consider the rich contextual information in the ranking list; and (ii) address the hidden issues underlying user behaviors, i.e., to mine observe pattern in queries without any click (non-click queries), and to model tracking logs which cannot truly reflect the user browsing intents (untrusted observation). Specifically, we adopt a recurrent neural network to model the contextual information and estimates the conditional likelihood of user feedback at each position. We then incorporate survival analysis techniques with the probability chain rule to mathematically recover the unbiased joint probability of one user's various behaviors. DRSR can be easily incorporated with both point-wise and pair-wise learning objectives. The extensive experiments over two large-scale industrial datasets demonstrate the significant performance gains of our model comparing with the state-of-the-arts.
Introduction
In search and recommendation systems, implicit feedback—primarily clicks—is the primary source of training data. However, this data is inherently biased. Users are disproportionately likely to click top-ranked items simply due to their visibility, a phenomenon known as position bias. Furthermore, traditional models often treat documents in a list as independent entities, ignoring the contextual influence of preceding items.
To address these challenges, researchers from Shanghai Jiao Tong University, Microsoft, and Alibaba introduced Deep Recurrent Survival Ranking (DRSR). This framework leverages survival analysis—a statistical method commonly used in clinical trials—to model user browsing behavior as a sequential process. By doing so, it accounts for position bias and contextual dependencies while extracting valuable signals from "non-click" sessions.
Methodology: Deep Recurrent Survival Ranking
DRSR treats the ranking list as a sequence and a user click as an "event." The model architecture consists of two primary components: a Recurrent Neural Network (RNN) backbone and a survival analysis integration layer.
The RNN Backbone
Unlike standard point-wise or pair-wise models that score items in isolation, DRSR uses an LSTM-based RNN to process the document list from top to bottom. This allows the model to capture the contextual information of the entire list, recognizing that a user's decision to click an item is influenced by the quality of the items they have already seen.
Survival Analysis Integration
In medical statistics, survival analysis models the time until an event occurs. DRSR maps this to the ranking domain: the "event" is a click, and "survival" represents the user continuing to browse the list.
The model defines two key metrics:
- Hazard Rate ($h_i$): The conditional probability that a user clicks document $i$, given that they have examined all documents up to that position.
- Survival Probability ($S_i$): The probability that the user has not clicked any document before position $i$ and is still actively observing the list.
Handling Censored Data and Non-Clicks
A major innovation of DRSR is its treatment of non-click queries as "censored" data. In survival analysis, if a study ends before an event occurs, the data is censored. In ranking, a session with no clicks implies that the user observed $L$ items without finding them relevant. DRSR uses these sessions to refine its understanding of document relevance, whereas many traditional unbiased methods simply discard them.
Experimental Evaluation
The authors evaluated DRSR on two large-scale datasets: the Yahoo Search Engine Dataset (~30k queries) and the Alibaba Recommender System Dataset (~179k queries). To simulate real-world conditions, biased click logs were generated from human-labeled relevance data using standard click models (PBM and CCM).
Key Results
DRSR consistently outperformed state-of-the-art baselines, including Unbiased LambdaMART and various DNN-based models, across metrics such as MAP and NDCG.
As shown in the results table, DRSR (using both pairwise and pointwise debiasing) achieved significant performance gains. On the Yahoo dataset, DRSR reached a MAP of 0.842, surpassing the best baseline of 0.829. The model also demonstrated high data efficiency, outperforming competitors even when trained on significantly smaller subsets of data.
Conclusion and Future Work
DRSR provides a robust framework for unbiased ranking by mathematically modeling the browsing process. By treating non-clicks as informative signals and accounting for list context, it recovers true relevance from noisy logs.
Future research directions include extending the model to handle multi-click sessions, where users may click several items in a single browsing sequence. Additionally, the authors suggest refining the model to distinguish between "good abandonment" (finding information without clicking) and "bad abandonment" (user frustration).
Summary updated