Finding the blogs that are consistently devoted to a topic, rather than single posts that mention it, is called feed distillation. The best systems for it bridge the vocabulary gap between two-word user queries and the way bloggers actually write by adding related terms mined from Wikipedia. The query "road cycling" gains "Lance Armstrong" and "UCI ProTour". This added vocabulary can be dramatic: Arguello et al. report a 372% gain in mean average precision (MAP) on that query. It is also erratic. The same paper finds that expansion hurts roughly a third of queries, because heavily linked, popular entities such as country names dominate the added terms. Ernsting et al. show expansion can even wash out the original keyword entirely, the "lactose lesson".
We propose that much of this damage comes from how the feed's language model is smoothed, and that it can be fixed there. Smoothing is the step that assigns nonzero probability to words a document never used, and it silently decides how much each query term matters. Retrieval systems smooth every query term the same way, whether the user typed it or an expansion step added it. We propose to treat the two groups differently. Original terms get only the length-correcting smoothing that suits short queries. Expansion terms get an extra discounting stage against a background that measures how many different feeds use a word. Under that background, terms spread across the whole blogosphere, which is what popular entities are, contribute little. Terms concentrated in a few sources, which are the useful jargon, contribute a lot.
Core Claim
The smoothing literature establishes that a single smoothing parameter has two jobs that conflict. The first is estimating a document's true word distribution from a finite sample, which depends on document length. The second is discounting common words, which acts like inverse document frequency. Two-stage smoothing (Zhai and Lafferty; Zhai and Lafferty, 2017) separates the jobs by applying Dirichlet smoothing followed by Jelinek-Mercer interpolation. Dirichlet smoothing adds pseudo-counts that shrink with document length. Jelinek-Mercer interpolation linearly mixes the document model with a background word distribution. Those papers find Dirichlet best for short title queries and Jelinek-Mercer best for long, verbose ones. An expanded feed query is both at once: a two-word title query plus a verbose tail of machine-chosen terms. We claim this mixed query needs term-specific smoothing. Writing $P_\mu(w|F) = (c(w,F) + \mu P(w|C)) / (|F| + \mu)$ for the Dirichlet-smoothed model of feed $F$, where $c(w,F)$ is the count of word $w$ in the feed, $|F|$ its length and $P(w|C)$ the collection word distribution, we score
$$ s(F,Q) = \sum_{w \in Q_o} q(w) \log P_\mu(w|F) \; + \; \beta \sum_{w \in Q_e} q(w) \log\big[(1-\lambda) P_\mu(w|F) + \lambda P(w|B)\big]. $$
Here $Q_o$ are the original terms, $Q_e$ the expansion terms, $q(w)$ each term's query weight, $\beta$ the usual weight on expansion, and $\lambda$ the second-stage mixing weight. The background $P(w|B)$ is proportional to the number of distinct feeds containing $w$, rather than to its raw token count. The second stage's discounting effect is strongest for words with high background probability. A feed-spread background therefore penalizes exactly the "appears everywhere" entities that cause popularity bias. It does not penalize a term that is frequent only because one prolific source repeats it.
What is new is the combination of three things: role-split smoothing inside a single query, a source-level background statistic, and its use to control expansion drift in feed distillation. No paper in our file applies two-stage smoothing or query-side discounting to expanded feed queries.
Research Program
The prior work plays three distinct roles here. Elsas et al. and Arguello et al. supply the retrieval setting and the expansion method: Wikipedia anchor-text expansion over large "feed as one concatenated document" representations, scored with single-stage Indri smoothing. Their per-query failures, together with the lactose example of Ernsting et al., motivate the proposal.
Smucker and Allan both motivates and constrains it. They showed that moving term discrimination into the query model, by weighting each query term with $-\log P(w|C)$, improved MAP by 6.4% on description queries. It also halved the optimal Dirichlet parameter (2500 to 1250), which is evidence that heavy smoothing had been compensating for missing term weighting. Their result means our method has to beat a query-side reweighting, not just plain smoothing. Shakery and Zhai show two-stage smoothing already helps in web topic distillation (0.064 versus 0.054 average precision over Dirichlet). However, they apply it uniformly and use a static token background.
We will build the scorer above on the TREC Blog06 collection in two feed representations. The first is the large-document model used by the best TREC 2008 CMU run. The second is the post-aggregation Blogger model of Weerkamp et al.. Using both tests whether the effect depends on the representation. We will reimplement Wikipedia anchor expansion so every system receives identical $Q_e$.
The analysis half matters as much as the scores. We will label each topic as helped or hurt by expansion under the single-stage baseline. We then measure how many hurt topics our method recovers and how many helped topics it keeps. We will also correlate each expansion term's change in contribution with its feed-spread, to check that the mechanism works for the reason claimed. As a generality check, we will swap the Wikipedia terms for terms extracted from language-model-generated passages in the style of InteR. That paper itself reports degradation from noisy expansion past two rounds, which is the same failure in a different form.
Baselines
The naive reference point is the unexpanded large-document query-likelihood model with Dirichlet smoothing. It shows how much expansion buys at all.
The strongest existing method is the large-document model with Wikipedia expansion and spam filtering, cmuLDwikiSP of Arguello et al.. It is the best TREC 2008 feed-distillation run in our file, with MAP 0.306 and P@10 0.434; without spam filtering (cmuLDwiki) it reaches MAP 0.302. It is the natural competitor because it uses the exact expansion source we aim to repair. For TREC 2007 topics, the strongest reported number is MAP 0.3556 for the two-stage Title+Full Blogger system of Weerkamp et al.. A further reference is the local-evidence model of Lee et al., at MAP 0.3031 on TREC 2008.
Because reported numbers come from different pipelines, we will report them but base conclusions on our reimplementations. Every method will use the same index, spam filter, and expansion terms. Each will get an equal grid budget over its parameters: $\mu$ and $\beta$ for the baseline, plus $\lambda$ for ours, with the baselines given extra grid points to match our total. Parameters will be tuned on the 2007 topics and tested on 2008, and the reverse.
Four ablations isolate the key ingredients. The first applies uniform two-stage smoothing to all terms with a token background, which is Zhai and Lafferty applied directly. The second is role-split smoothing with a token background, which isolates the background choice. The third is role-split smoothing with a feed-spread background, which is our full method. The fourth replaces the second stage with Smucker–Allan style weighting of the expansion terms, $-\log$ of the feed-spread probability, which isolates smoothing from reweighting.
Beating the baselines means two things. First, a MAP gain over the reimplemented cmuLDwikiSP that is statistically significant (paired test over topics) in both transfer directions. Second, a reduction in the number of expansion-hurt topics without losing the large per-query gains on topics like "road cycling".
Why not just…?
Why not just tune the expansion weight β lower? Lowering $\beta$ shrinks all expansion terms equally, so it trades the "road cycling" gains for protection against country-name drift. It cannot separate a useful term from a harmful one. Our per-query helped/hurt analysis tests this directly: if a tuned-$\beta$ baseline recovers as many hurt topics while keeping helped ones, the proposal fails.
Why not just use Smucker and Allan's query-side weighting? This is the real threat. Smucker and Allan already obtain a significant 6.4% gain by weighting terms rather than changing smoothing. Their weight is computed from token frequency, whereas popularity bias concerns how many sources use a word. That points to the background statistic as the likely active ingredient, rather than smoothing versus reweighting. Our fourth ablation settles it. If feed-spread query weighting matches role-split smoothing, we will report the contribution as the source-level statistic applied to expansion terms, which is simpler and just as useful.
Why not just predict which queries to expand and skip the rest? Selective expansion is a binary decision per query. The failures documented by Arguello et al. and Ernsting et al. come from a few bad terms inside otherwise useful expansions, so skipping the whole query discards good terms along with bad ones. We will report an oracle selective-expansion upper bound: per topic, the better of the expanded and unexpanded runs. If our method approaches this bound, the per-term approach is justified.
Why not just use a dense dual encoder? The LinkedIn system of Ramanujam et al. (Recall@10 0.4242) matches members to items, not topics to persistently devoted sources. It also requires fine-tuning a 3B-parameter model. We have no reported feed-distillation number for dense retrievers: no reported result found on Lacuna for dense retrieval on TREC Blog feed distillation. We will include an off-the-shelf dense retriever over feed representations as a reference point, but not as the target of this claim.
Proposed Experiments
The main experiment runs all systems on TREC Blog06 with the 2007 (45 topics) and 2008 topic sets, reporting MAP and P@10 under cross-year tuning.
The decisive check is the per-topic accounting. Our method should convert a substantial share of the roughly one-third of expansion-hurt topics into neutral or positive outcomes. It should retain at least most of the gain on the topics expansion helps.
The mechanism check tests the proposed cause directly. The loss in contribution of each expansion term should rise with its feed-spread, and country or organization entities should be among the most discounted. The lactose-style check measures how often an original term falls out of the top-contributing terms, which should drop to near zero because original terms bypass the second stage.
Two robustness checks follow. First, repeat the comparison with language-model-generated expansion terms. Second, repeat it under the post-aggregation Blogger model, to show the effect is not an artifact of the large-document representation.
A secondary check follows Smucker and Allan. If expansion-term discrimination has truly moved into the second stage, the optimal Dirichlet $\mu$ for the original terms should fall toward values suited to short title queries.
References
- Document and Query Expansion Models for Blog Distillation
- Language Modeling Approaches to Blog Postand Feed Finding
- A study of smoothing methods for language models applied to information retrieval
- A Study of Smoothing Methods for Language Models Applied to Ad Hoc Information Retrieval
- Retrieval and Feedback Models for Blog Distillation
- Lightening the load of document smoothing for better language modeling retrieval
- Relevance Propagation for Topic Distillation UIUC TREC 2003 Web Track Experiments
- Blog feed search with a post index
- Synergistic Interplay between Search and Large Language Models for Information Retrieval
- KLE at TREC 2008 Blog Track: Blog Post and Feed Retrieval
- Large Scale Retrieval for the LinkedIn Feed using Causal Language Models