Financial and statistical reports store their numbers in hierarchical tables, where a cell means something only through its chain of headers ("2024 → Q3 → Operating expenses"). Language models answer numerical questions about these tables poorly. The main reason is not that they cannot do the arithmetic. They fail to find the right numbers under the nested headers, and when they do find them, they compute unreliably. Both failures are documented only in English. This proposal asks what happens when the question is asked in Swahili, Bengali, or Turkish. We hypothesize that most of the cross-lingual drop on these tasks comes from finding numbers, not from computing with them. If that is right, the fix belongs in a small, targeted component and not in the reasoning model.
We propose to build that component and test the hypothesis. A multilingual cross-encoder (a model that reads the question and one candidate together and outputs a relevance score) scores every cell of a document. Each cell is rendered as its full header path. The cross-encoder selects the operands the question needs. A language model then writes a short program that refers to those operands only by slot name, and an external executor computes the answer. The transformer never touches a digit during the calculation. The program-writer never sees a multilingual table schema. The language-sensitive step reduces to matching a non-English phrase against an English or local header path, which is the kind of fine-grained lexical alignment cross-encoders handle well.
Core Claim
The claim has two parts. The first is diagnostic. We expect that the gap between English and lower-resource-language accuracy on hierarchical numerical table QA is mostly an extraction gap. If gold operands are supplied, the gap should shrink sharply. This would extend the English-only "extraction bottleneck" finding of Evaluating LLMs' Mathematical Reasoning in Financial Document Question Answering to the cross-lingual setting. That paper shows models fail mainly at locating numbers buried under multi-level headers, and do far less badly at calculating once the numbers are found.
The second part is constructive. A header-path cross-encoder trained to select operands, and fed the target-language query directly, should close more of that gap than two alternatives. One is translating the query to English first. The other is handing the whole table to a reasoning model.
What is new is the combination of three things. The first is operand-level re-ranking over hierarchical paths. The second is a cross-lingual setting. The third is an execution path that keeps arithmetic and multilingual code generation away from the transformer. No paper in our file combines these.
Relation to Prior Work
Three papers motivate the proposal by locating the bottleneck. MultiHiertt builds a benchmark from S&P 500 reports and proposes MT2Net. MT2Net linearizes each cell into a full header-path sentence, scores those sentences with a first-stage fact retriever, and generates a symbolic program. MT2Net reaches 36.22% EM, and 64% of its errors are retrieval failures. We keep its header-path linearization and its symbolic execution. We replace its independent fact scoring with joint question–path cross-attention, and we move the whole problem into non-English queries.
The EEDP study cited above supplies the extraction-bottleneck diagnosis, in English only. HiTab supplies hierarchy-aware logical forms and cell-alignment annotations. It also shows accuracy collapsing to 14.1% at nesting depth five or more. That depth stratification is one we will reuse.
Two papers constrain the design. M3TQA is the only multilingual table QA work in the file, covering 97 languages. It finds that free-form thinking traces help high-resource languages and degrade or diverge for low-resource ones, where direct supervised fine-tuning is needed. This is why our language-sensitive component is a supervised discriminative scorer rather than a reasoning trace. M3TQA also feeds the gold table in context, uses mostly flat or matrix layouts, and does not separate extraction from computation. Those are the three things we change.
E⁵ has a language model explain the headers, write Pandas code, and execute it. It reaches 85.08 EM on HiTab with GPT-4. This shows that external execution works. However, the program depends on the model correctly naming multilingual schema elements in code, and our program-writer avoids that dependency by referencing slots only.
Three papers supply the method. The cross-encoder architecture, where the question and candidate are concatenated and attend to each other in every layer, comes from A Survey of Model Architectures in Information Retrieval and Cross-Encoder Re-ranking in Two-Stage Retrieval Pipelines. The idea of moving arithmetic out of attention comes from TaCube, which pre-computes aggregates and ranks them into context. TaCube lifts TAPEX on TAT-QA from 49.6 to 66.2 F1, but it is limited to first-order operations on flat English tables. The precision ceiling reported in Can Transformers Solve Least Squares to High Precision? is one more reason not to let the model compute.
Beyond Linearization is the closest graph-side competitor. It re-ranks row–header–value triples with question-guided PageRank and is robust to row shuffling. It is English-only and is not designed for multi-operand arithmetic.
Research Program
Build the evaluation set. We start by constructing a cross-lingual hierarchical numerical QA set, because none exists. We take the MultiHiertt and HiTab test questions that have gold supporting cells. We have professional translators render a stratified sample, a few hundred per language, into roughly six languages spanning high, mid, and low resource levels. Tables stay in their original English.
We do not use machine translation for the test set. Machine-translated questions would quietly favor the translate-then-answer baseline, because translating them back recovers something close to the original English.
Two further settings use M3T-Bench's professionally annotated numerical-computation and cell-extraction items:
- a setting where both the question and the table are in a non-English language;
- a corpus setting, where each gold table is hidden among distractor tables from the same benchmark, so retrieval is not trivial.
Training data uses machine-translated questions over the MultiHiertt and HiTab training splits. Human-translated items are reserved for test.
The selection model. Each cell becomes a header-path string that includes its top-header and left-header ancestors. This follows MT2Net's linearization, with HiTab's tree structure supplying the paths. A multilingual cross-encoder scores each question–path pair. The target is whether the cell is a gold operand, trained with a contrastive loss against hard negatives. Hard negatives are sibling cells that share all but one ancestor. These are exactly the cells a flat matcher confuses.
A cheap first stage keeps the top few hundred cells per document, using a multilingual dense or lexical retriever over the path strings. The cross-encoder then re-ranks them.
Execution. The top-$k$ operands are shown to an instruction-tuned language model as labelled slots ("A: [path] = 2,894"). The model emits a program in a small operation language of add, subtract, multiply, divide, average, and ratio, referring only to slot letters. An executor substitutes the values and computes the answer. The model is never asked to produce a digit of the result.
Baselines
The naive reference point is direct prompting. A strong instruction-tuned model receives the target-language question plus the whole document's tables flattened to Markdown, with no retrieval and no execution. This measures the raw cross-lingual gap and is run on every language and split.
The strongest existing pipeline in the file is E⁵ with GPT-4, which reports 85.08 EM on English HiTab (E⁵). On MultiHiertt the strongest fine-tuned number is MT2Net at 36.22 EM and 38.43 F1 (MultiHiertt). We found no reported cross-lingual result on Lacuna for either benchmark, and no per-task numbers for M3TQA's fine-tuned Llama-3.1-8B.
We will rerun E⁵ in two ways: with target-language questions, and wrapped in Cross-Lingual-Thought prompting. The latter is the XLT method of Not All Languages Are Created Equal in LLMs, which raised gpt-3.5-turbo on MGSM from 23.3 to 70.0. Combining XLT with E⁵ is the most natural strong competitor. We will also rerun MT2Net's retriever, trained on the same machine-translated questions we use.
Translate-test runs in two versions. In the first, the query is machine-translated to English and an English-trained version of our own pipeline is applied. In the second, the translated query is fed to E⁵.
Fairness comes from four controls:
- every method uses the same language model for program writing or answering;
- every method gets the same machine-translated training data;
- every learned component gets equal hyperparameter-search budgets;
- every method gets the same top-$k$ operand budget where it applies.
The ablations isolate the key ingredients:
- a flat cell string ("row label | column label | value") versus the full header path;
- a multilingual bi-encoder versus the cross-encoder, holding the training data fixed;
- the cross-encoder's operands passed to the model, which then computes in-text, versus executor computation;
- an oracle condition with gold operands supplied.
The oracle separates extraction error from computation error. Beating the strongest competitor means two things. In the low-resource languages, our pipeline must close a larger share of the English-to-target EM gap than XLT+E⁵ does. It must do this without losing English accuracy relative to E⁵ run in English.
Why not just…?
Why not just translate the question into English and use an English system? This is the real threat. XLT's MGSM gains show that pivoting through English works well for word problems. Our counter-hypothesis is narrow. Machine translation corrupts domain labels such as "operating income" versus "operating profit," or localized unit words, and those labels are exactly what header-path matching depends on. A cross-encoder that sees the original phrasing can match it to the header directly.
We will settle this empirically. We stratify test items by how many header terms the question mentions and by measured translation fidelity, and we report translate-test against direct scoring in each stratum. We also run a dual-input cross-encoder that sees both the original and the translated query. If translate-test matches direct scoring across strata, we will report that the extraction gap is a translation-solvable problem. In that case the diagnostic decomposition becomes the main contribution.
Why not just give a long-context reasoning model the whole table, or let E⁵ write code? Reasoning models do help on English hierarchical tables. DeepSeek-R1 reaches 70.91 EM on RealHiTBench fact checking (RealHiTBench). The evidence against relying on them comes from three places:
- the same benchmark reports collapse beyond 20,000-token tables;
- reasoning accuracy falls from about 70% to 58% as MultiHiertt questions require more steps (EEDP study);
- M3TQA finds that free-form thinking diverges for low-resource languages.
The XLT+E⁵ baseline tests this alternative directly. The corpus setting with distractor tables is where we expect it to break.
Why not just fine-tune a multilingual dense retriever and skip the cross-encoder? This is cheaper, and if it works it should be preferred. The bi-encoder ablation answers the question directly. Our reason to expect a gap is that sibling cells differ in a single ancestor token. A single-vector encoding compresses away exactly that difference, while joint attention can compare it token by token. We will report the cost and accuracy tradeoff.
There is one acknowledged risk for both retriever types. Conan-Embedding-v2 notes that text encoders struggle to distinguish quantities. Questions with numeric conditions ("the year sales exceeded 2,000") are therefore reported as a separate slice.
Decisive Experiments
The first check is the gap decomposition. For each language, we compare EM under direct prompting, under the pipeline with predicted operands, and under the pipeline with gold operands. The extraction hypothesis predicts that the gold-operand condition leaves a small residual gap. Most of the drop should sit between predicted and gold operands. If instead the gold-operand gap stays large, the bottleneck is program writing in the target language, and the proposal's central premise fails.
The second check measures operand recall@k for the gold supporting cells, stratified by language, by HiTab nesting depth, and by the term-density strata above. The header-path cross-encoder should hold up at depth five or more, where HiTab reports 14.1%. It should also beat both translate-test and the flat-string ablation in low-resource languages.
The third check is end-to-end EM against XLT+E⁵ and MT2Net, in two settings: with the gold table given, and in the corpus setting with distractor tables. The headline number is the fraction of the English-to-low-resource gap that each method closes.
References
- Evaluating LLMs' Mathematical Reasoning in Financial Document Question Answering
- MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Data
- HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation
- M3TQA: Massively Multilingual Multitask Table Question Answering
- E⁵: Zero-shot Hierarchical Table Analysis using Augmented LLMs via Explain, Extract, Execute, Exhibit and Extrapolate
- A Survey of Model Architectures in Information Retrieval
- DKITNLP at ArchEHR-QA 2025: A Retrieval Augmented LLM Pipeline for Evidence-Based Patient Question Answering
- TaCube: Pre-computing Data Cubes for Answering Numerical-Reasoning Questions over Tabular Data
- Can Transformers Solve Least Squares to High Precision?
- Beyond Linearization: Attributed Table Graphs for Table Reasoning
- Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting
- RealHiTBench: A Comprehensive Realistic Hierarchical Table Benchmark for Evaluating LLM-Based Table Analysis
- Conan-Embedding-v2: Training an LLM from Scratch for Text Embeddings