Semantic Serialization of Tabular Data for Language Models

98 concepts85 papers1 proposal

Traditional machine learning for tabular data relies on rigid feature engineering and specialized architectures like gradient-boosted trees. While effective, these methods often struggle with heterogeneous data formats, missing values, and "low-shot" scenarios where training data is scarce. A growing body of research proposes a different approach: transforming structured tables into natural language text. By serializing rows into descriptive sentences, researchers can leverage the vast semantic knowledge and reasoning capabilities of Large Language Models (LLMs) to solve non-linguistic tasks.

The Mechanism of Serialization

The core idea is to bridge the gap between structured records and the prose-based training data of LLMs. Instead of feeding a model a vector of numbers, the data is "verbalized." As Hegselmann et al. and Wang et al. demonstrate, this typically involves mapping column headers and cell values into a structured sentence format or a sequence of key-value pairs.

For example, a medical record with columns for Age, Smoker, and Diagnosis might be serialized into: "The patient is 45 years old, has a history of smoking, and was diagnosed with hypertension."

This transformation can take several forms:

  • Template-based: Using predefined natural language templates to wrap values, as seen in the work of Chen et al. and Milecki et al..
  • Markup-based: Representing tables using Markdown, JSON, or CSV formats to preserve structural relationships while remaining text-centric [Freire et al.; Gao et al.].
  • Generative Narratives: Using an LLM to synthesize raw attributes into coherent synthetic personas or "pseudo-notes" [Lin et al.; Lee et al.].

Why Semantic Grounding Works

The intuition behind serialization is that feature names (headers) carry significant semantic weight. Traditional models treat a column named "Oxygen Saturation" as an abstract numerical vector. In contrast, by converting this into text, Qiu et al. and Shysheya et al. show that the model can apply its pre-trained understanding of medical context to interpret the value.

This semantic grounding provides several advantages:

  1. Handling Missing Data: LLMs can use learned contextual relationships to maintain performance even when specific data points are absent, bypassing the rigidity of traditional imputation [Xing et al.; Wu et al.].
  2. Zero-Shot Transfer: Because the model understands the meaning of the features, it can perform predictions on entirely new datasets without task-specific fine-tuning. Gardner et al. demonstrate this with TABULA-8B, a foundation model that treats spreadsheet rows like sentences to achieve zero-shot capabilities.
  3. Cross-Modal Integration: Serialization allows structured data to be fused with unstructured text (like doctor's notes) or images through a unified text-based interface [Wu et al.; Milecki et al.].

Beyond Classification: Cleaning and Generation

Serialization is not limited to prediction. By treating relational tables as text, data cleaning becomes a text-to-text transformation task. Zhu et al. and Zhang et al. argue that this allows models to identify semantic inconsistencies—such as a city not matching its zip code—that traditional rule-based systems might miss.

Furthermore, the process can be reversed for data generation. Borisov et al. introduced GReaT, which uses LLMs to generate realistic synthetic tabular data by learning the distribution of serialized text strings. This enables "arbitrary conditioning," where users can generate missing values based on any subset of features.

Limits and Trade-offs

Despite the flexibility of serialization, it is not a universal replacement for traditional machine learning. Dinh et al. show that while LLMs (via the LIFT framework) can match specialized baselines like XGBoost on low-dimensional data, they face challenges in high-stakes quantitative fields.

Pierrehumbert et al. and Kostrzewa et al. find that for complex financial forecasting and large-scale imbalanced data, traditional gradient boosting models remain more accurate and efficient. Additionally, Liu et al. warn that serializing data into text can cause LLMs to inherit and amplify social biases present in their pre-training corpora, potentially making them less "fair" than traditional statistical models.

Key Foundational Work

Table-based Fact Verification with Self-adaptive Mixture of Experts (2022) — contributed 1 concept elements to this direction.
TabLLM: Few-shot Classification of Tabular Data with Large Language Models (2022) — contributed 1 concept elements to this direction.
Toward a stable and low-resource PLM-based medical diagnostic system via prompt tuning and MoE structure (2022) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_natural_language_data_serialization_144fe7d5,
  title = {natural language data serialization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/natural-language-data-serialization/rdr_144fe7d5064f4cefbcb6559e7fbd5e20},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_natural_language_data_serialization_144fe7d5_v1,
  title = {natural language data serialization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/natural-language-data-serialization/rdr_144fe7d5064f4cefbcb6559e7fbd5e20/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake