Cell-Level Hybrid Retrieval for Nested Tables of Single-Nucleotide RNA Modifications

Biologists now map chemical marks on RNA, such as $N^6$-methyladenosine (m6A, a methyl group attached to one adenine), at the level of individual bases. They report the results in large nested tables. Rows are grouped by gene, then transcript, then base position. Columns are grouped by tissue, then normal versus cancer, then sample or timepoint. Answering even a simple question ("is the site at this base methylated in liver tumors but not normal liver?") means finding two or three specific cells among tens of thousands. Each of those cells is only meaningful through its chain of parent headers. Today's language models fail at exactly this lookup step. On financial tables, the dominant error is pulling the wrong cell, not doing the wrong arithmetic (Evaluating LLMs' Mathematical Reasoning in Financial Document Question Answering). Reasoning collapses once a table exceeds about 20,000 tokens (RealHiTBench) or once headers nest five levels deep (HiTab).

We propose to change what gets retrieved. Current systems retrieve whole tables, or flattened rows, and ask the model to find the cell. Instead, we retrieve individual cells, each indexed with its full row-header path and column-header path. Each cell is scored by three signals. The first is exact matching on identifiers such as gene symbols, transcript IDs and mark names. The second is semantic matching on the header path, so that "writer knockdown" can match "METTL3 shRNA." The third is a position-aware score that understands genomic coordinates. That third signal is where single-nucleotide resolution becomes a retrieval problem. A query about base 15,400,201 should match a base-resolution row at that base. It should match only weakly a row at 15,400,203. It should match, with low confidence, a 150-nucleotide antibody-enrichment peak that merely contains the base. Neither word-matching nor embedding retrievers make these distinctions. The retrieved cells are then reassembled into a small sub-table that keeps its header tree, and a language model answers only from that sub-table.

Core Claim

Our central claim is about where the error comes from. On large nested tables of base-level modification data, most remaining errors come from retrieving the wrong cell, and a retriever can fix them if it works on cells, keeps the header structure, and understands coordinates. Concretely, scoring each cell with header-path-aware hybrid matching plus a resolution-aware position term should do two things. It should flatten both the depth cliff and the table-size cliff on end-to-end question answering. It should also resolve adjacent-base and peak-versus-site confusions that defeat both pure lexical and pure dense retrieval.

The score of cell $c$ for query $q$ is

$$s(q,c) = \alpha\, S_{\text{lex}}(q, \text{ids}(c)) + \beta\, S_{\text{dense}}(q, \text{path}(c)) + \gamma\, \exp\!\big(-d(q,c)/\tau\big)\, r_c,$$

where the terms are defined as follows:

  • $S_{\text{lex}}$ is a BM25-style score over the identifier tokens in the cell and its headers.
  • $S_{\text{dense}}$ is an embedding similarity between the query and the ordered header path, encoded level by level rather than as one concatenated string.
  • $d(q,c)$ is the distance in nucleotides between the queried base and the cell's site. It is zero if the base falls inside a peak interval.
  • $\tau$ is a tolerance scale.
  • $r_c$ equals 1 for base-resolution entries and $w_0/w_c$ for a peak of width $w_c$ relative to a reference width $w_0$. A wide peak therefore never outranks an exact site.

We expect the weights $\alpha,\beta,\gamma$ to be learned per query type. Both ingredients have precedents. Ma et al. learn hybrid score combinations (Hybrid First-stage Retrieval Models for Biomedical Literature), and THYME uses a mixture of field experts (Tailoring Table Retrieval from a Field-aware Hybrid Matching Perspective).

Relation to Prior Work

Three lines of work supply the parts but not the combination.

Hybrid retrieval supplies the method. Luan et al. showed that single-vector dense encoders lose discriminative margin on long inputs, and that interpolating with BM25 helps (MRR@10 rises from 0.302 to 0.343 on MS MARCO) (Sparse, Dense, and Attentional Representations for Text Retrieval). Ma et al. showed that dense encoders collapse on biomedical identifiers: BioASQ 8 Batch 2 MAP is 0.1173 for the dense encoder versus 0.2718 for BM25. Their concatenated hybrid with reranking reached 0.4056 average MAP. Both operate on free text and on documents. We reuse their scoring machinery on cells.

THYME is the closest existing work. It applies field-aware sparse/dense matching to tables. But it retrieves whole flat Wikipedia tables, and it pools headers in a way that has no notion of parent-child nesting. We retrieve cells inside nested tables, and we add a coordinate term that THYME has no reason to need.

Hierarchical table reasoning motivates and constrains the design. TUTA (TUTA) and HyTrel (HyTrel) show that tree or hypergraph structure must be kept explicitly. TreeThinker gets large gains by having the model draw the header tree first. These methods all still place the whole table in front of the model, which is exactly what fails beyond 20,000 tokens.

AIT-QA is a constraint on our design (AIT-QA). It shows that concatenating nested headers into one string confuses identical sub-headers under different parents. This is why our dense channel encodes header paths level by level.

TABALIGN (From Table to Cell) also motivates the design. It found that in 67% of errors the model skipped its planned cell-grounding step. It also found that annotators agree on which cells are relevant far more often (93%) than on whether a reasoning step is correct (73%). That supports making cell selection an explicit, separately supervised stage.

The modification-detection literature supplies the application regime. Nanopore callers (RedNano; Coral) produce true base-level calls. Weakly supervised models (WeakRM; m6A-TSFinder in m6A-TSHub) infer sites inside 100–200 nucleotide peaks. As a result, real tables mix the two resolutions. m6A-TSHub catalogs 184,554 normal and 499,369 cancer sites, including 177,998 at base resolution, across 23 tissues. It also reports that 93.25% of modification-affecting cancer variants act in a single tissue. Questions over such data are therefore almost always conditional on a deep header path. None of these papers addresses querying the resulting tables.

Research Program

We will build three things.

A benchmark of nested modification tables. We will export hierarchical tables from m6A-TSHub's site and variant records. Rows will nest as gene → transcript → site, with base-resolution and peak-level entries mixed. Columns will nest as tissue → normal/cancer → sample. We will add supplementary-style tables patterned on published layouts, such as JEV peaks across 24 and 48 hours post-infection (Yao et al.). Questions will be generated from templates for lookup, comparison across header branches, aggregation over a subtree, and variant gain/loss. An LLM will then paraphrase them, and humans will audit a sample. Every question carries its gold cell set. We will deliberately build three kinds of hard cases:

  • adjacent-base distractors, with sites at $p\pm1$ to $p\pm5$;
  • resolution distractors, where a peak contains the queried base but a nearby base-level site does not;
  • sibling-header distractors, where the same sample label appears under different tissues.

The cell retriever. Each cell becomes a record holding its value, its identifier tokens, its level-by-level header paths, and, for coordinate cells, a parsed interval. The model is trained with matching-score dropout as in THYME, so that no single channel dominates. Following Ma et al., we will use synthetic queries generated from the tables, because labeled queries will be scarce.

The assembly and answering step. The top-$k$ cells are merged into a minimal sub-table, made of the union of their header paths, and rendered as HTML. HTML is used because RealHiTBench found structured text beats images. An LLM answers and must cite the cell IDs it used. A citation outside the retrieved set counts as a grounding violation. This lets us measure plan bypass directly.

To test whether the gains are specific to genomics, we will also run the retriever on HiTab and RealHiTBench. There, the coordinate term is switched off and only the header-path hybrid remains.

Baselines

The naive reference point is a frontier LLM given the full table serialized as HTML or LaTeX, with no retrieval. Truncation is reported where the context overflows. On RealHiTBench, the reported number for GPT-4o with LaTeX input is 60.31% exact match on Fact Checking. This baseline shows how much the size and depth cliffs cost on our benchmark.

The strongest structural reasoning baseline is TreeThinker prompting on the same model. It lifted GPT-4o Chart Generation Pass@1 from 14.29% to 33.55% on RealHiTBench. On HiTab, the strongest reported result in the file is hierarchy-aware logical forms at 45.1% execution accuracy. We will treat that figure as the HiTab bar to beat.

The strongest retrieval baseline is row-chunk hybrid retrieval. Each row is linearized with its concatenated headers and indexed with BM25 plus dense interpolation, in the style of ME-HYBRID and Ma et al.'s HybridRerank (0.4056 average MAP on BioASQ 8, a document-level number). Alongside it we will run THYME adapted to return sub-tables. THYME reports NDCG@5 of 65.72 on NQ-TABLES at table level. We found no reported result on Lacuna for cell-level retrieval on hierarchical tables, nor for any method on epitranscriptomic table QA. For these settings, the comparison is internal.

All retrievers will share the following controls:

  • the same synthetic training queries;
  • the same encoder backbone;
  • the same top-$k$ token budget passed to the same answering LLM;
  • the same hyperparameter-search budget, fixed in GPU hours.

Ablations isolate the key ingredients. We will remove the position term. We will replace level-by-level header encoding with concatenated header strings. We will drop the lexical or the dense channel. We will also give the reasoner the gold cells, to obtain an oracle-retrieval ceiling.

Beating the baselines means three things. First, higher cell recall@k at matched token budget. Second, higher end-to-end accuracy on tables above 20,000 tokens and at depth five or more. Third, a clear reduction in adjacent-base and peak-versus-site errors relative to row-chunk hybrid retrieval. Matching it on easy lookups is acceptable.

Why not just…?

Why not just put the whole table in a long-context model with TreeThinker-style prompting? Because both cliffs are documented on exactly this kind of input. RealHiTBench reports collapse beyond about 20,000 tokens. HiTab reports 14.1% accuracy at depth five or more. Transcriptome-wide site tables exceed both limits routinely. Our size- and depth-stratified results will show whether a newer model closes the gap. If it does on small tables, we will report that our method matters only above the size threshold.

Why not just use row-level hybrid RAG with concatenated headers? This is the real threat, because it is cheap and already strong on text. AIT-QA's sibling-header confusion and the 30–35% paraphrase failure rate suggest that concatenated headers will mislead it. Its lexical channel also treats "15,400,201" and "15,400,203" as unrelated tokens, while its dense channel treats them as nearly identical. We expect the gap to appear mainly on our distractor splits. If row-chunk hybrid retrieval matches us there, then the header-path and position terms are not earning their cost, and we will say so.

Why not just normalize the tables into a relational database and have an LLM write SQL with interval joins? For a curated portal like m6A-TSHub, this is often the right tool. The hard part with literature tables is the normalization itself: deciding which nested header becomes which column, and doing so under paraphrased questions. HiTab's best logical-form system reaches only 45.1%. We will run text-to-SQL over automatically normalized tables, and also over oracle-normalized tables. If oracle-normalized SQL beats us by a wide margin, then the problem is really normalization, and we would refocus the project on header-path extraction as a normalizer. The cell records we build are already a normalized form.

Proposed Experiments

Retrieval quality. We will measure cell recall@k and NDCG@10 against gold cell sets on the new benchmark. Results will be stratified by table size (below and above 20,000 tokens), header depth (one to four versus five or more), and distractor type.

End-to-end accuracy. We will measure exact match on answers and the grounding-violation rate, the latter as a direct analogue of TABALIGN's plan-bypass measurement.

Generality. We will run the coordinate-free variant on HiTab and RealHiTBench, compared against the reported 45.1% and 60.31% figures, under matched answering models where we rerun them.

Decisive checks. Three outcomes would settle the claims:

  • The no-position-term ablation should lose most of its accuracy on adjacent-base and resolution distractors while staying level on header-only questions. That would show that coordinate-aware matching, not hybridization in general, does the work.
  • Accuracy above the 20,000-token threshold should stay within a few points of accuracy below it, whereas the full-table baseline should drop.
  • The gap between our retriever and oracle retrieval should be small relative to the gap between full-table reasoning and oracle retrieval. That would confirm that retrieval, not reasoning, was the bottleneck.

A negative result on the third check would be informative too. It would mean that nested modification tables fail at the reasoning stage, which redirects effort toward verifiers like TABALIGN's.