Table Retrieval May Not Necessitate Table-specific Model Design

Zhiruo Wang, Zhengbao Jiang, Eric Nyberg, Graham Neubig

Abstract

Tables are an important form of structured data for both human and machine readers alike, providing answers to questions that cannot, or cannot easily, be found in texts. Recent work has designed special models and training paradigms for table-related tasks such as table-based question answering and table retrieval. Though effective, they add complexity in both modeling and data acquisition compared to generic text solutions and obscure which elements are truly beneficial. In this work, we focus on the task of table retrieval, and ask: "is table-specific model design necessary for table retrieval, or can a simpler text-based model be effectively used to achieve a similar result?" First, we perform an analysis on a table-based portion of the Natural Questions dataset (NQ-table), and find that structure plays a negligible role in more than 70% of the cases. Based on this, we experiment with a general Dense Passage Retriever (DPR) based on text and a specialized Dense Table Retriever (DTR) that uses table-specific model designs. We find that DPR performs well without any table-specific design and training, and even achieves superior results compared to DTR when fine-tuned on properly linearized tables. We then experiment with three modules to explicitly encode table structures, namely auxiliary row/column embeddings, hard attention masks, and soft relation-based attention biases. However, none of these yielded significant improvements, suggesting that table-specific model design may not be necessary for table retrieval.

Introduction

Tables are ubiquitous in digital documents, from Wikipedia to financial reports, and contain information that unstructured text often lacks. Because tables possess a two-dimensional structure of rows and columns, the research community has generally assumed that retrieving the correct table for a query requires "table-aware" models. These specialized architectures often incorporate complex features like auxiliary row/column embeddings, structure-aware attention masks, and specialized pre-training. However, these designs add significant complexity to model development and data acquisition. This paper investigates a fundamental question: Is this complexity actually necessary for table retrieval, or can a simple, well-trained text retriever perform just as well?

Methodology: Text vs. Specialized Architectures

The researchers compared two primary paradigms for table retrieval:

  1. Dense Passage Retriever (DPR): A standard, "off-the-shelf" text retriever based on BERT. To handle tables, it "linearizes" them—converting the table into a continuous string of text using delimiters (e.g., Title. Header1 | Header2. Row1Cell1 | Row1Cell2...).
  2. Dense Table Retriever (DTR): A specialized model designed specifically for tabular data, featuring cell-wise position resets and explicit structural embeddings.
The training and architectural pipelines for the general-purpose DPR versus the specialized DTR.

To further test the necessity of structure, the authors experimented with "Explicit Structure Injection," adding three common table-specific modules—auxiliary embeddings, hard attention masks, and soft attention biases—to the standard DPR to see if they provided any performance boost.

Datasets and Structural Analysis

The primary benchmark used was NQ-table, a subset of the Natural Questions (NQ) dataset containing approximately 12,000 questions answerable via Wikipedia tables.

A typical Wikipedia table from the NQ-table dataset, illustrating the type of structured data that can be linearized for text-based retrieval.

A manual analysis of 100 random samples from this dataset revealed a critical insight: in over 70% of cases, a table could be correctly identified simply by matching keywords in the question to the table's title or headers. Only about 21% of cases required understanding specific row/column alignments, suggesting that for retrieval, semantic content often outweighs structural complexity.

Key Insights and Results

  • Implicit Learning Beats Explicit Design: A standard DPR model, when fine-tuned on linearized tables, learns to recognize the "shape" of a table implicitly. It does not require a dedicated column index to understand that cells separated by delimiters belong to different fields.
  • The Importance of Delimiters: Experiments showed that delimiters (like the | symbol) are more important than the actual sequence of data. These symbols act as structural markers that allow the transformer's attention mechanism to parse the layout without a custom architecture.
  • DPR Outperforms Specialized Models: The fine-tuned, text-based DPR (DPR-table) achieved a Top-1 retrieval accuracy of 67.91%, while the state-of-the-art specialized model (DTR-large) reached only 63.98% under identical conditions.
  • Zero-shot Prowess: Even without seeing tables during training, the standard text-trained DPR was surprisingly competitive, proving that general language understanding transfers effectively to tabular data.

Conclusion and Future Work

The findings suggest that for the task of table retrieval, table-specific model design may be unnecessary. A strong text-based retriever, combined with proper linearization and fine-tuning, provides a simpler and more effective solution. However, the authors note that while retrieval may not require structural awareness, more complex tasks like Table Question Answering—which involves logic and math across cells—may still benefit from specialized architectures. Future work will explore the limits of text-based models on massive, highly technical, or sparse tables where headers may be less descriptive.

Summary updated

Abstract

Tables are an important form of structured data for both human and machine readers alike, providing answers to questions that cannot, or cannot easily, be found in texts. Recent work has designed special models and training paradigms for table-related tasks such as table-based question answering and table retrieval. Though effective, they add complexity in both modeling and data acquisition compared to generic text solutions and obscure which elements are truly beneficial. In this work, we focus on the task of table retrieval, and ask: "is table-specific model design necessary for table retrieval, or can a simpler text-based model be effectively used to achieve a similar result?" First, we perform an analysis on a table-based portion of the Natural Questions dataset (NQ-table), and find that structure plays a negligible role in more than 70% of the cases. Based on this, we experiment with a general Dense Passage Retriever (DPR) based on text and a specialized Dense Table Retriever (DTR) that uses table-specific model designs. We find that DPR performs well without any table-specific design and training, and even achieves superior results compared to DTR when fine-tuned on properly linearized tables. We then experiment with three modules to explicitly encode table structures, namely auxiliary row/column embeddings, hard attention masks, and soft relation-based attention biases. However, none of these yielded significant improvements, suggesting that table-specific model design may not be necessary for table retrieval.
BibTeX
@misc{wang2022table,
  author = {Zhiruo Wang and Zhengbao Jiang and Eric Nyberg and Graham Neubig},
  title = {{Table Retrieval May Not Necessitate Table-specific Model Design}},
  year = {2022},
  eprint = {2205.09843},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.09843},
}

Loading related content...

Figure