Many multi-hop questions fail at the second search, not the first. To answer "In what city was the director of Inception born?", a system finds the Inception article easily. The passage it needs next ("Christopher Nolan was born in Westminster, London…") shares almost no words with the question. Current systems handle this second hop in one of three ways. They squeeze the question plus the first passage into a single dense vector. They ask a language model to write a new query. Or they generate document identifiers token by token. Each way has a cost: the vector loses detail, the language model invents entities, and token-by-token generation is slow.
We propose a cheaper alternative. A small latent-variable model reads the question and the first-hop passage and predicts a distribution over the words the next passage will contain. The latent state is sparse and non-negative, a "topic" vector in which most entries are exactly zero. Each sample from it decodes into a weighted bag of words drawn only from the corpus's own vocabulary. Drawing several samples gives several distinct, interpretable hypotheses about what the bridge passage is about. Each hypothesis becomes a keyword query, so expansion stays grounded in words that actually exist in the index.
Core Claim
The claim has two parts. The first is about retrieval. Uncertainty about the next hop is usually multimodal: the bridge passage might be a biography, a filmography, or a place article. A sparse latent captures these modes as separate samples, whereas a deterministic predictor averages them into a blurry, dense term vector. The blurring argument is the one the sparse-prior literature makes against Gaussian priors in topic models: ATM and SawETM both show that dense priors smear mass across unrelated themes.
The second part is about keeping the latent informative cheaply. Posterior Collapse and Latent Variable Non-identifiability proves that a latent collapses, meaning it stops carrying information about its input, exactly when distinct latent values produce the same output distribution. That paper enforces a one-to-one decoder with gradients of convex networks, which requires second-order derivatives. For a topic decoder the output is a linear map of a non-negative vector. That map is one-to-one on the simplex whenever the topic-word matrix has full column rank. Identifiability therefore reduces to a rank condition that a cheap penalty can enforce.
This connects to a familiar topic-model failure. When topics are duplicated or junk, the latent has directions that change nothing in the output. Sampling then looks diverse in latent space but produces identical queries. We expect the rank condition to be what turns latent sampling into real query diversity.
The obvious combination at this intersection would attach a variational latent to an autoregressive chain-of-thought rewriter. The file's material on decoder-driven collapse (Bowman et al.) predicts that such a latent would be ignored, because a strong text decoder can model the query from its own context. We deliberately avoid that regime. Our decoder sees only the latent and emits a bag of words. A context-conditioned decoder appears only as an ablation that should reproduce collapse.
Research Program
The model is a conditional variational autoencoder, a latent-variable model trained to reconstruct a target from a compressed code while keeping that code close to a learned prior. Its components are defined as follows:
- $c$ is the context: the question plus the retrieved hop-1 passage.
- $d$ is the gold hop-2 passage, represented as a bag of words over a capped corpus vocabulary of $V$ index terms.
- $z \in \mathbb{R}_{\ge 0}^{K}$ is the bridge-topic vector.
- The prior $p_\psi(z \mid c)$ is a Gamma distribution whose parameters are produced by a small transformer encoder.
- The training-time posterior $q_\phi(z \mid c, d)$ is a Weibull distribution that also sees $d$.
- The decoder is $p(d \mid z) = \mathrm{Mult}(\Phi z / \lVert \Phi z \rVert_1)$, where $\Phi \in \mathbb{R}_{\ge 0}^{V \times K}$ is the topic-word matrix.
We reuse the Weibull reparameterization and the closed-form Weibull-to-Gamma divergence from WHAI. The training objective is
$$\mathcal{L} = \mathbb{E}_{q_\phi(z\mid c,d)}\big[\log p(d\mid z)\big] - D_{\mathrm{KL}}\big(q_\phi(z\mid c,d)\,\Vert\,p_\psi(z\mid c)\big) + \lambda \log\det\big(\bar\Phi^{\top}\bar\Phi + \epsilon I\big),$$
with two additional symbols:
- $\bar\Phi$ is $\Phi$ with columns normalized to unit length.
- $\lambda$ weights a log-determinant term that pushes the topics toward linear independence. This term is the cheap stand-in for the one-to-one decoder of the identifiability paper. Its cost is one $K \times K$ determinant per step, with no gradients of gradients.
At test time we draw $S$ samples from the prior. Each sample yields a term-weight vector $\Phi z$. We keep its top terms, add the salient entities of the hop-1 passage, and issue a weighted BM25 query. The $S$ ranked lists are merged by reciprocal-rank fusion. For three- and four-hop MuSiQue questions, the procedure repeats with the newly retrieved passage appended to $c$.
The design supplies, at the hop level, what Generative Multi-hop Retrieval argues dense vectors lack: a structured representation of the intermediate state. GMR decodes document strings and is reported to be about 24 times slower than vector search. Our decoding is one matrix-vector product per sample.
Three analyses accompany the system. The first measures latent health: active units, the effective rank of $\Phi$, and topic redundancy. We compare these with and without the rank penalty, and with a Gaussian-latent softmax decoder in place of the Weibull/Gamma pair. The second tests whether sample diversity is real: we compute the pairwise overlap of top terms across samples and check whether distinct samples retrieve distinct gold candidates. The third verifies that the topics are interpretable as bridge relations such as birthplace, filmography, or administrative region. If they are, they can be inspected when a chain goes wrong. That property addresses the error propagation IRCoT identifies as the main failure of sequential pipelines.
Baselines
The naive reference point is one-shot retrieval with the raw question, run with both BM25 and a dense retriever. It is also run in a "hop-1 entity" variant, which appends the named entities of the first retrieved passage to the question. This variant is the cheapest possible bridge, and a large share of any gain we report must survive comparison with it.
The strongest dense competitor is Momentum Posterior Regularization for Multi-hop Dense Retrieval. MoPo trains a retriever that sees only the query history to imitate a teacher retriever that sees language-model-written summaries of the gold evidence. It reports Retrieval EM@100 of 82.20 on HotpotQA, against 79.70 for Multi-hop Dense Retrieval (MDR). In zero-shot transfer to StrategyQA it reports 39.24 against 36.06 for MDR.
MoPo is the closest prior work because it also learns a test-time "prior" that mimics a training-time "posterior" with privileged access to later hops. It does this in dense vector space, with summaries from a language model. We will train it and our model on the same HotpotQA training split with the same first-hop retriever. We give both the same hyperparameter-search budget measured in GPU-hours. MoPo's summary-generation cost is reported separately, not hidden. Beating MoPo means exceeding 82.20 EM@100 on HotpotQA under this protocol. Failing that, it means matching MoPo while adding measurable gains when the two are fused, which would show the two approaches capture different evidence.
For downstream answers we use one fixed reader across all systems. Our reference points are MoPo's reported QA F1 of 64.3, BeamAggR at 62.9, and IRCoT at 56.2, all as reported in the MoPo paper. IRCoT is rerun with the same reader to keep the comparison fair. On MuSiQue, no isolated retrieval number for IRCoT, MoPo, or MDR was found on Lacuna; we will report our own reruns.
The lexical-rewriting competitor is a compact query rewriter trained with ranking feedback following RaFe, which reports 2–3% QA accuracy gains on NQ, TriviaQA, and HotpotQA. We adapt it to the second hop by feeding it the question plus the hop-1 passage. Its backbone size is matched to our encoder.
Two ablations isolate the key ingredients. The first isolates the latent: a deterministic sparse predictor of hop-2 term weights, with the same encoder, a non-negative output, and an L1 penalty. This ablation is fed the same total number of query terms as the $S$ samples combined. The second isolates identifiability: our model without the log-determinant term, and our model with a decoder that also receives $c$.
Why not just…?
Why not just let a language model write the next-hop query, as IRCoT does? This is the most widely used alternative and it does close the multi-hop gap: IRCoT reports gains of 11–21 points in retrieval recall. Its costs are sequential large-model calls and invented content. RaFe and InteR both document that unconstrained expansion drifts toward entities absent from the index, and InteR finds that more than two rounds hurts. Our expansions can only contain index terms. Still, IRCoT may beat us on raw recall. We will report recall against latency and a hallucinated-term rate, the fraction of query terms absent from any gold passage. A loss on recall at a large latency advantage is a legitimate outcome. A loss on both would falsify the approach.
Why not just use pseudo-relevance feedback, expanding the query with the top words of the retrieved hop-1 passage? This classic trick is free and is our "hop-1 entity" baseline in a richer form. Its weakness is structural: the hop-1 passage describes the film, while the bridge passage describes the director. The words needed (birth, born, raised) are often absent from hop 1. The comparison that settles this question is a breakdown by question–hop-2 content-word overlap. The finding by Yang et al. that many answer passages share no content words with the question tells us where to look. If pseudo-relevance feedback matches us in the zero-overlap bucket, the learned latent is not earning its place.
Why not just train a deterministic sparse term predictor, with no latent variable at all? This is the real threat. Such a predictor shares our encoder, our grounding in index terms, and our speed. The only thing it lacks is the ability to keep several hypotheses apart. If bridge uncertainty is mostly unimodal, it will tie us, and the variational machinery and sparse prior would be unnecessary. The ablation is built to decide this question. We hold the total term budget fixed. We then measure recall@k as the number of samples $S$ grows, separately on questions where the hop-1 entity has many candidate relations (MuSiQue's branching compositions) and few. A tie under these conditions would lead us to drop the latent and keep the simpler model.
Proposed Experiments
Training uses the supporting-paragraph annotations of HotpotQA, with retrieval over the full Wikipedia corpus. The main multi-hop test is MuSiQue. MuSiQue's construction removes single-hop shortcuts, and a single-hop model drops 30 F1 there relative to HotpotQA. StrategyQA serves for zero-shot transfer, where MoPo's 39.24 EM@100 is the bar. Metrics are Retrieval EM@100 and recall@k per hop, answer and supporting-fact F1 with a fixed reader, latency per question, and index size. For the latent itself we report active units, effective rank of $\Phi$, and cross-sample term overlap.
Four checks decide the proposal.
The first check concerns the lexical gap. Our gain over MoPo and over pseudo-relevance feedback should be concentrated in the low-overlap buckets. If gains are uniform across buckets, the lexical-gap story is wrong.
The second check concerns multimodality. Recall should rise with $S$ for the variational model and stay flat for the deterministic ablation at matched term budget. Without this, the sparse latent adds nothing.
The third check concerns identifiability. Removing the log-determinant term should lower the effective rank of $\Phi$, raise duplicate topics, and shrink real cross-sample diversity. Giving the decoder access to $c$ should drive active units toward zero. Together these would show, in a retrieval setting and at first-order cost, the link between collapse and non-identifiability that Posterior Collapse and Latent Variable Non-identifiability proved for Gaussian latents. The Gaussian softmax variant should also yield denser, less separable topics, as the sparse-prior work predicts.
The fourth check is a falsification test on the decoder design. If even the context-free decoder collapses on HotpotQA, the bag-of-words design is not enough to prevent collapse. We would then test the warm-up and residual scaffolding of SawETM before drawing conclusions.
References
- ATM: Adversarial-neural Topic Model
- Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network
- Posterior Collapse and Latent Variable Non-identifiability
- Generating Sentences from a Continuous Space
- WHAI: Weibull Hybrid Autoencoding Inference for Deep Topic Modeling
- Generative Multi-hop Retrieval
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
- Momentum Posterior Regularization for Multi-hop Dense Retrieval
- RaFe: Ranking Feedback Improves Query Rewriting for RAG
- Synergistic Interplay between Search and Large Language Models for Information Retrieval
- WikiQA: A Challenge Dataset for Open-Domain Question Answering
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
- ♫ MuSiQue: Multihop Questions via Single-hop Question Composition