Some question-answering systems skip whole documents and search directly over billions of short text spans, returning the answer phrase itself. They must solve two problems at once. The first is choosing which spans to store at all, since a Wikipedia-scale corpus contains tens of billions of possible substrings. The second is how to describe each stored span so that a question can find it, even when the question and the answer use different words. Current phrase indexes handle the first problem with a learned keep-or-discard filter. They handle the second by attaching to every span a keyword vector computed over its whole surrounding paragraph. That paragraph-wide keyword vector blurs together every candidate in the paragraph. Many of those candidates are distractors that sit next to the question's entity but answer a different question.
We propose using a sentence's grammatical structure for both decisions. Spans are proposed at constituent boundaries; a constituent is a grammatical unit such as a noun phrase or clause in a parse tree. Each span's keyword vector is then restricted to the words of the clause that grammatically governs it, instead of the whole paragraph. The claim is that syntax supplies a better unit of retrieval than either a learned token filter or a fixed window. It keeps more gold answers per gigabyte of index, and it attaches exact-match evidence to the right span within a paragraph. This second effect is a specific, testable way of reducing the vocabulary-mismatch damage that keyword matching suffers when evidence is spread across a paragraph.
Core Claim
Two prior findings motivate the proposal. A Constituent-Centric Neural Architecture for Reading Comprehension found that about 70% of SQuAD answers are exact parse-tree constituents and about 97% lie within four words of a constituent boundary. However, it used this only to restrict answer extraction inside a single given passage, with no retrieval. Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index (DENSPI) indexes every span of up to 20 tokens and prunes with a binary classifier. Each span is represented by a dense vector from BERT (a learned embedding capturing meaning) concatenated with a sparse tf-idf vector (word counts weighted by rarity), and the sparse part is computed over the span's broad context. The result needs a 1.2TB index.
Our claim has two parts, and each could fail on its own. The first concerns which spans get stored. Constituents plus a small set of boundary expansions should dominate a learned token filter on gold-answer coverage at every index-size budget. The second concerns what each span's keyword vector contains. Restricting it to the governing clause should improve exact match over paragraph-scoped sparse vectors. The gain should concentrate on questions whose gold paragraph contains several entity-adjacent distractor spans.
Neither part is the obvious "constituent-filtered DENSPI." The first is a budgeted comparison against learned filtering, not a replacement claim. The second, syntax-scoped lexical evidence for phrase retrieval, does not appear in any work in our file. The closest relatives each leave it open. Salient Phrase Aware Dense Retrieval (SPAR) trains a dense model to imitate BM25 at the passage level, with no notion of phrase units. Bridging the Training-Inference Gap for Dense Phrase Retrieval filters tokens statistically and has no sparse component.
Research Program
We first parse the Wikipedia corpus once, offline. Parsing cost counts toward indexing time, not query time, because questions are never parsed. This sidesteps the main domain-mismatch concern about parsers: they degrade on queries and informal text, but encyclopedic prose is close to the treebank text they were trained on.
From each parse we propose all constituents under the 20-token cap. To recover the roughly 27% of answers that are near-constituents but not exact ones, we add expansions that move a boundary by up to four tokens. We then rank all proposals with a small learned scorer, the same kind of start/end classifier DENSPI uses. Finally we keep the top spans until we reach a fixed index budget. So syntax proposes the candidates and the learned scorer chooses among them. Because the parser alone never decides, a single parse error does not delete the right answer unless the true span is also more than four tokens from any constituent. We will measure that residual ceiling directly.
For representation, each kept span gets DENSPI's dense start/end vectors unchanged. Its sparse vector is built from the tokens of the smallest clause that dominates it in the parse. We add the clause's governing predicate where the clause is a relative or embedded clause, and we fall back to the sentence when no clause node exists. The dense and sparse scores are combined by a single learned weight, as in DENSPI, so the only change is where the sparse evidence comes from. The intuition is concrete. For a question like "who designed the bridge over X," the paragraph mentions X once, but the paragraph also contains three person names. Paragraph tf-idf gives all three names the same "X" credit. Clause-scoped tf-idf gives that credit only to the name in the clause that mentions X.
The analysis is stratified by how badly the question's words fail to match. Following the observation by Yang et al. that many correct answers share no content words with the question, we bucket questions by content-word overlap between the question and the gold answer's clause. We expect no sparse benefit, and possibly a small cost, in the zero-overlap bucket, where only the dense vector can help. We expect the largest benefit in the partial-overlap bucket where the gold paragraph has several distractors. Finding that pattern is part of the claim.
Baselines
The naive reference point is DENSPI as published: all spans up to 20 tokens, a classifier filter, and paragraph-level sparse vectors. Its reported result is 33.4 exact match on SQuAD-Open at 0.51 seconds per query, with a 1.2TB index (DENSPI). We will retrain it in our codebase on the same encoder, corpus snapshot, and training questions, so that it is not penalized by implementation drift. Beating it means higher exact match at an equal or smaller index.
The strongest competitor on the pruning question is the learned token filter from Bridging the Training-Inference Gap for Dense Phrase Retrieval. That method cut index size by over 70% while acting as a denoiser, and reported gains of 2.7 points in top-1 phrase accuracy on NQ and 2.0 on TriviaQA. The file gives these only as relative gains; no absolute exact-match figure for that system is reported on Lacuna. We will run its filter on our encoder at three matched budgets, 30%, 15%, and 7% of the full span set. It gets the same hyperparameter-search budget as our scorer. Beating it means higher gold-answer coverage and exact match at every matched budget, not just one.
The ablation that isolates the second ingredient keeps our span set fixed and swaps only the sparse scope. Three variants are compared: paragraph scope, a symmetric window of plus or minus k tokens, and clause scope. We tune k over a grid and also match the average number of nonzero sparse terms across the three, so clause scope cannot win simply by being shorter. A second ablation removes the syntactic proposals and feeds the learned scorer all spans, to test whether syntax adds anything beyond the scorer.
Why not just…?
Why not just use a learned filter and skip parsing? This is a fair challenge, and the budget sweep answers it. The case for syntax is that it supplies a candidate set with a known coverage profile, the 70/97 distribution, before any training. It also gives a structural signal that flat models miss; the CCNN ablation lost 5.8 exact-match points when its tree model was replaced with a sequential one (CCNN). If the learned filter matches our coverage curves at all budgets, we will report that syntax gives nothing for span selection and keep only the scoping result.
Why not just use a fixed window around each span for the sparse vector? This is the real threat. A ±k window may capture most of what a clause captures at far lower engineering cost. The matched-sparsity window ablation settles it. We predict clause scope wins mainly on long sentences with embedded clauses, where a window crosses into a neighboring clause's entities. We will report results split by sentence length so that a null effect is visible as such.
Why not just do passage-level hybrid retrieval and run a reader? Interpolating BM25 with a dense retriever is simple, and BM25 is robust out of domain (BEIR). Separate pipelines cost latency, though: SPAR reports about 58ms for DPR plus BM25 versus about 20ms when the two are unified. A retrieve-then-read pipeline also adds a reader pass, and DENSPI reported 35 seconds per query for DrQA. We include a BM25-plus-dense passage retriever with the same reader as a reference. The question is not whether the pipeline is more accurate, since it may be, but whether phrase indexing closes the gap at a fraction of the latency.
Proposed Experiments
We run on SQuAD-Open over the DENSPI Wikipedia dump and on open-domain Natural Questions and TriviaQA. For each we report exact match, top-k phrase accuracy, index size, and per-query latency on identical hardware.
The first decisive check is the coverage-versus-budget curve. It plots the fraction of dev questions whose gold answer string survives as an indexed span, for constituents plus expansions versus the learned token filter, from 5% to 40% of all spans. If our curve is not strictly above the filter's at small budgets, the pruning half of the claim fails.
The second decisive check is the matched-sparsity scope ablation, stratified by question-clause lexical overlap and by the number of distractor spans in the gold paragraph. The scoping claim is supported only if clause scope beats both paragraph scope and the best tuned window in the partial-overlap, high-distractor stratum. It should do so without losing in the zero-overlap stratum.
Finally, we measure the parser-induced recall ceiling: the gold answers lying more than four tokens from any predicted constituent. We compare it on SQuAD, whose paragraphs are curated, against NQ, whose paragraphs are less so. This tells future users how much the method depends on parse quality.
References
- A Constituent-Centric Neural Architecture for Reading Comprehension
- Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index
- Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?
- Bridging the Training-Inference Gap for Dense Phrase Retrieval
- WikiQA: A Challenge Dataset for Open-Domain Question Answering
- BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models