Linearizing Tabular Data for Language Models

23 concepts22 papers

Transformers have revolutionized natural language processing by treating text as a continuous, one-dimensional sequence of tokens. However, much of the world's structured knowledge resides in two-dimensional grids: tables. A table relies on spatial layout—rows, columns, and hierarchical headers—to convey meaning. Forcing a standard language model to read a table creates a fundamental modality mismatch.

To bridge this gap without inventing entirely new model architectures, researchers use table linearization (or serialization). As defined by Zhai et al., linearization is a representational strategy that flattens two-dimensional tabular structures into a continuous one-dimensional string of text. This simple transformation allows standard transformer architectures to process structured data as a sequential natural language task.

The Two Paradigms of Flattening

How exactly do we turn a grid into a line? The literature broadly splits into two methodologies: structural serialization and template-based translation.

Structural Serialization (Delimiters and Key-Values)

The most direct way to flatten a table is to preserve its raw structural associations using special tokens or text delimiters. For instance, Liang et al. demonstrate that sequence-to-sequence models can process tables effectively when rows and columns are separated by simple characters like newlines and pipes (|).

Alternatively, tables can be encoded as a sequence of explicit key-value pairs. Narayan et al. show that pairing attribute labels (headers) directly with their corresponding cell values allows foundation models to leverage the semantic context of column headers for data wrangling tasks. To handle more complex multi-row layouts, Nan et al. introduce special structural tokens to explicitly demarcate row and column boundaries, embedding the table's topology directly into the token sequence.

Template-Based Translation (Pseudo-Sentences)

Instead of exposing the model to raw structural syntax, another school of thought converts tabular data into synthetic natural language prose. In this paradigm, cell values are injected into predefined natural language templates to form "pseudo-sentences."

For example, Chen et al. utilize template-based embedding to let pre-trained models exploit their existing grasp of syntax and semantics when verifying facts against tables. Similarly, Yu et al. apply templates to individual rows and columns to unify tabular data into a shared "language space" alongside unstructured text. This strategy is widely used across specialized domains, from public health fact-checking (Akhtar et al.) to biomedical question answering (Luo et al.) and cross-lingual inference (Agarwal et al.). To ensure the model respects the layout during generation, Chen et al. combine template serialization with a multi-pass deliberation mechanism that refines the output text.

Why It Works: The Power of Self-Attention

Why does flattening a table actually work? It seems counterintuitive that a model designed for linear text can navigate a complex relational grid. The answer lies in the mechanics of the transformer's self-attention mechanism.

When a table is linearized, every cell and header becomes a token in a unified sequence. In structural serialization, keeping the header close to the value or using explicit key-value pairs ensures that the model can easily map attributes to values. In fact, Gupta et al. found that preserving these direct structural associations as key-value pairs is often more effective than forcing the data into synthetic natural language sentences. Synthetic prose can introduce distracting linguistic fluff, whereas key-value serialization allows the self-attention mechanism to directly calculate dependencies between an attribute and its value without syntactic interference.

Furthermore, because the entire flattened table is fed into the model simultaneously, the attention mechanism can dynamically correlate distant cells. As Nan et al. note, a unified attention mechanism allows the model to attend to both an incoming question and the entire table structure at the same time, dynamically reconstructing the 2D relationships needed for multi-cell reasoning.

Pretraining and Format Agnosticism

One might worry that the specific choice of linearization format (e.g., whether you use pipes, JSON-like structures, or row-centric vs. column-centric layouts) would severely limit a model's generalization. However, Zhang et al. discovered that while formatting choices significantly impact smaller models, extensive multitask pretraining makes larger language models increasingly robust and agnostic to the specific method of table linearization.

This robustness has enabled a unified approach to tabular tasks. Toolkits like the one developed by Garanina et al. capitalize on this by providing consistent wrappers that flatten diverse datasets into uniform linear sequences, eliminating the need for custom, task-specific architectures. Whether the task is expanding cryptic column names (Zhang et al.), explaining model predictions (Menon et al.), or generating scientific text grounded in numerical data (Guo et al.), linearizing the table allows the model to treat structured data wrangling as a standard text-to-text generation problem.

Key Foundational Work

INFOTABS: Inference on Tables as Semi-structured Data (2020) — contributed 2 concept elements to this direction.
TabFact: A Large-scale Dataset for Table-based Fact Verification (2020) — contributed 1 concept elements to this direction.
TableGPT: Few-shot Table-to-Text Generation with Table Structure Reconstruction and Content Matching (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_table_linearization_da849575,
  title = {table linearization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/table-linearization/rdr_da8495755ca547a8b2e21b59a8d9dc95},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_table_linearization_da849575_v1,
  title = {table linearization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/table-linearization/rdr_da8495755ca547a8b2e21b59a8d9dc95/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}

See something wrong? Report a mistake