Abstract
Recency is an important dimension of relevance for real-time Twitter search as users tend to be interested in fresh news and events. By incorporating various sources of evidence, the application of learning to rank (LTR) algorithms to real-time Twitter search has shown beneficial in finding not only relevant, but also recent tweets in response to given queries. However, the potential effectiveness brought by LTR may not have been fully exploited due to the lack of labeled data available for properly learning a ranking model, since human labels are expensive in real-world applications. To this end, this paper proposes a transductive algorithm that incrementally aggregate the labeled tweets through an iterative process. Experimental results on the standard Tweets11 dataset show that our approach is able to outperform strong baselines without the use of human labels.
Introduction
In the domain of real-time Twitter search, relevance is inherently tied to recency. Users seek the latest updates on emerging events, making traditional static ranking models less effective. While Learning to Rank (LTR) algorithms can effectively combine diverse features—such as social authority, content richness, and temporal data—they typically require substantial amounts of human-labeled training data. In a fast-paced microblogging environment, obtaining these labels is often prohibitively slow and expensive.
This paper addresses the "cold start" problem by proposing a transductive learning algorithm (TLTR) that automatically generates training data. By iteratively "self-labeling" search results, the system builds a ranking model without requiring any initial human intervention.
The TLTR Algorithm
The core of the proposed approach is the Transductive Learning for Twitter Ranking (TLTR) algorithm. Unlike inductive learning, which aims for a general model, transductive learning focuses on optimizing performance for a specific set of test data—in this case, the tweets retrieved for a particular query.
The TLTR process operates through an iterative loop of pseudo-labeling and model refinement. It begins by using a content-based retrieval model to obtain an initial set of tweets. The top-ranked tweets are assumed to be positive examples, while the bottom-ranked ones are treated as negative. An LTR model is then trained on this initial set and used to re-rank the remaining unlabeled data. This process repeats, gradually expanding the training set with the most confident predictions from the model.
Experimental Setup
The researchers evaluated TLTR using the Tweets11 dataset, a standard benchmark from the TREC 2011 Microblog track containing approximately 13.4 million tweets. The evaluation focused on 50 official search topics.
To represent the tweets, a multi-faceted feature set was employed including content-based relevance (BM25, KLIM), content richness (length, URLs), recency (time difference), authority (retweet counts), and tweet-specific features (hashtags, mentions).
Results and Discussion
The primary evaluation metric was Precision at 30 (P30). The experiments compared TLTR against a strong content-based baseline, KLIMQE, which achieved a P30 of 0.3782.
As shown in the results, the TLTR approach significantly improved performance across various underlying LTR algorithms, including RankSVM, ListNet, and LambdaMART. Specifically, TLTR combined with LambdaMART reached a P30 of 0.4212, representing an 11.37% improvement over the baseline without using any human labels.
A key insight from the study is the "self-correcting" nature of the transductive process. Even if initial pseudo-labels contain noise, the iterative selection of high-confidence examples allows the model to refine its understanding of relevance. Furthermore, by assigning higher weights to fresher tweets during the labeling process, the model explicitly prioritizes the real-time nature of the search task. Remarkably, TLTR often outperformed models trained on small sets of actual human-labeled data, likely because the transductive process produces a larger training set that more accurately reflects the specific distribution of the current search results.
Conclusion
The TLTR algorithm demonstrates that effective ranking models for real-time search can be built without the bottleneck of human annotation. While the current implementation requires manual tuning of the iteration count ($N$), future work could involve developing adaptive halting criteria to monitor training set purity. Additionally, exploring semi-supervised paradigms like co-training could further enhance the robustness of the self-labeling process.
Summary updated