Table Linearization for Sequence to Sequence Learning

This direction is split into other directions: table linearization. Its last version stays readable and citable here.

Version 1 of 1162 papers180 conceptsfrozen 2026-10-04

The fundamental tension in modern natural language processing is that while our most powerful models—Transformers—are designed to process one-dimensional sequences, the world’s data often lives in two-dimensional structures. Tables, database schemas, and document layouts contain rich relational information that is lost when simply read from top-to-bottom, left-to-right.

To bridge this gap, researchers have converged on a strategy called linearization (or serialization): the art of flattening structured data into a text string that preserves its underlying semantics. By transforming a table into a "pseudo-sentence," we can leverage the massive pre-trained linguistic knowledge of models like GPT, T5, and BERT to reason over structured data without inventing entirely new architectures.

The Mechanics of Flattening

The simplest way to linearize a table is to treat it as a series of attribute-value pairs. As explored in TableGPT by Heng Gong and colleagues, this involves using natural language templates to map data into a format compatible with sequence-to-sequence models. For example, a row about a city might become: "City: Tokyo | Population: 14 Million."

However, simple templates often fail to capture the complex spatial relationships of larger tables. To solve this, researchers use specialized structural markers. In Text-to-Table, Xueqing Wu and Jiacheng Zhang use unique tokens to denote cell and row boundaries, allowing the model to "see" the grid within a continuous stream of text. Similarly, MultiTabQA uses markers for table boundaries and columns to help Andrew Yates and his team perform operations like joins and intersections across multiple relational tables.

Preserving Semantic Context

The primary challenge of linearization is ensuring that a value (e.g., "14 Million") remains tethered to its header ("Population") even after the table is flattened.

One effective technique is header-cell pairing. In the development of KO-TaBERT, Changwook Jun and Jooyoung Choi prepended row and column headers to every single cell value. This redundancy ensures that the model maintains the relational context of the data regardless of where it appears in the sequence.

TaBERT, introduced by Graham Neubig and Pengcheng Yin, takes this further by concatenating the natural language question with column headers and data types. This creates a "joint understanding" where the model can align the user's intent directly with the table's schema.

Why Linearization Works

It might seem counterintuitive to take a structured table and turn it into "messy" text, but this approach works because it exploits the attention mechanism of Transformers.

As MUSTAFA CANIM and Michael R. Glass demonstrate in their work on capturing row and column semantics, standard Transformers can learn to associate headers with values through self-attention if the delimiters are consistent. By using newline separators or specific markers, the model learns a "syntax" of the table. This allows general-purpose models to perform complex reasoning—such as the numerical comparisons in numericNLG or the scientific reasoning in SciGen—without requiring specialized 2D neural layers.

Beyond the Table

The success of linearization has led to its adoption in increasingly diverse domains:

  • Document Understanding: DocVQA by Dimosthenis Karatzas and Minesh Mathew flattens 2D document layouts into 1D strings to answer questions about visual forms.
  • Robotics: G-PlanET represents physical environments as "object tables" (listing 3D coordinates and orientations), allowing Bill Yuchen Lin to treat robotic planning as a text-reasoning task.
  • Scientific Discovery: As surveyed by Xiusi Chen and Yu Zhang, researchers are now sequentializing DNA, molecular structures, and climate data, treating the entire natural world as a language to be decoded.

By reducing structure to sequence, linearization provides a universal interface between the rigid world of databases and the fluid reasoning of language models.

Papers in this version

Showing 60 of 162 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_table_linearization_for_sequence_to_sequ_728b2469,
  title = {Table Linearization for Sequence to Sequence Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/table-linearization-for-sequence-to-sequence-learning/rdr_728b2469ea8a4077aeafbbd15359cb19},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_table_linearization_for_sequence_to_sequ_728b2469_v1,
  title = {Table Linearization for Sequence to Sequence Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/table-linearization-for-sequence-to-sequence-learning/rdr_728b2469ea8a4077aeafbbd15359cb19/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}