DTT: An Example-Driven Tabular Transformer by Leveraging Large Language Models

Arash Dargahi Nobari, Davood RafieiarXiv 2023 · arXiv · OpenReview

Abstract

Many organizations rely on data from government and third-party sources, and those sources and organizations do not follow the same data formatting. This introduces challenges in integrating data from multiple sources. Commercial database systems do not offer adequate support for integrating data from heterogeneous sources, and manual integration is both time-consuming and inefficient. While state-of-the-art approaches rely on similarity functions and textual transformations, they often fail to handle challenging cases where multiple mappings are required, or the mappings go beyond simple textual transformations. In this paper, we study the potential of deep neural models for transforming tables for joinability. In particular, we cast the problem as a prediction task and develop a framework that leverages large deep-learning language models to transform tabular data from a source formatting to a desired target representation. Our framework can efficiently learn the pattern for mapping the source formatting into the expected target using just a few examples, which can then be used for table joining, filling in missing values, and error detection. Compared to state-of-the-art mapping and joining approaches, our framework delivers noticeably more accurate and scalable performance on both real-world and synthetic datasets. Our experimental evaluation also shows that the performance of the proposed framework using our fine-tuned model is at par or better than large language models such as GPT-3, despite the significant difference in size, and that integrating large language models into our framework improves their performance.

Introduction: The Challenge of Tabular Data Integration

Data integration is often described as the "janitor work" of data science. When merging tables from disparate sources—such as government records and internal databases—researchers rarely find matching formats. For instance, one table might list a name as "Jocelyne Thomas," while another uses a specific identifier like "j.thomas."

Figure 1: Examples of source strings and their corresponding target transformations for joinability.

Traditional solutions generally fall into two categories: manual rule-writing, which is unscalable, or string-similarity functions, which often fail on complex transformations like extracting initials and changing case. Recent machine learning approaches frequently struggle with "noise" (data errors) or require exhaustive searches through a massive space of possible transformation rules. The Deep Tabular Transformer (DTT) aims to overcome these hurdles by learning transformation patterns on the fly from just a handful of examples.

The DTT Framework

DTT moves away from rigid string-matching rules in favor of a generative approach. The framework is built on several key architectural pillars:

  1. Byte-Level Modeling: Unlike standard Large Language Models (LLMs) that process "tokens" (groups of characters), DTT utilizes ByT5, a model operating at the byte/character level. This is critical for tabular data where IDs, codes, and partial names do not follow natural language patterns.
  2. Decomposition & Serialization: To handle long tables and LLM input limits, DTT breaks the problem down into multiple "sub-tasks." Each sub-task contains a few source-target examples and one new row to transform.
  3. Few-Shot Learning via Fine-Tuning: The model is fine-tuned on a massive synthetic dataset of string transformations (concatenation, casing, substrings, etc.). This teaches the model the underlying concept of transformation rather than specific data values.
  4. Aggregator: Because LLMs can be inconsistent, DTT runs multiple trials for each row using different example pairs. An aggregator then uses Maximum Likelihood Estimation (MLE) to select the most frequent or probable output, effectively filtering out "hallucinations" or errors caused by noisy examples.
Figure 2: The DTT pipeline, illustrating the flow from raw table data through decomposition, tokenization, model inference, and final aggregation.

Experimental Evaluation

The researchers evaluated DTT against three real-world benchmarks and several synthetic ones, including Web Tables (WT) from Google Fusion Tables and spreadsheet datasets from Microsoft Excel forums.

Key Results

  • Superior Accuracy: On the real-world Web Tables benchmark, DTT achieved an F1-score of 0.95, significantly outperforming the previous state-of-the-art (CST at 0.71).
  • Handling Complex Tasks: In the "String Reversal" (Syn-RV) test—a task that baffles most traditional join tools—DTT maintained an F1-score of 0.63, while baselines dropped to near zero.
  • Efficiency and Scalability: DTT scales linearly with row length. While traditional methods see execution times explode as strings lengthen, DTT remains performant, completing tasks in seconds that took other systems minutes.
  • Noise Robustness: Even when 80% of the provided examples were incorrect, DTT's F1-score only dropped by approximately 0.25, demonstrating the effectiveness of its aggregation logic.

Discussion and Future Work

A primary insight of this research is that characters matter more than words in tabular contexts. By using a byte-level model, DTT captures the precise syntax of data transformations that subword tokenizers (like those in GPT) often mangle. Furthermore, the study proves that a specialized, smaller model (582-million parameter ByT5-base) can match or exceed the performance of massive general-purpose models like GPT-3 for this specific niche.

Currently, DTT excels at syntactic transformations (character manipulation) but is less effective at semantic transformations requiring external world knowledge (e.g., mapping an ISBN to an author). Future work will focus on extending DTT to handle multi-column joins, automatic error detection, and exploring federated learning to protect sensitive private data during training.

Summary updated

Abstract

Many organizations rely on data from government and third-party sources, and those sources and organizations do not follow the same data formatting. This introduces challenges in integrating data from multiple sources. Commercial database systems do not offer adequate support for integrating data from heterogeneous sources, and manual integration is both time-consuming and inefficient. While state-of-the-art approaches rely on similarity functions and textual transformations, they often fail to handle challenging cases where multiple mappings are required, or the mappings go beyond simple textual transformations. In this paper, we study the potential of deep neural models for transforming tables for joinability. In particular, we cast the problem as a prediction task and develop a framework that leverages large deep-learning language models to transform tabular data from a source formatting to a desired target representation. Our framework can efficiently learn the pattern for mapping the source formatting into the expected target using just a few examples, which can then be used for table joining, filling in missing values, and error detection. Compared to state-of-the-art mapping and joining approaches, our framework delivers noticeably more accurate and scalable performance on both real-world and synthetic datasets. Our experimental evaluation also shows that the performance of the proposed framework using our fine-tuned model is at par or better than large language models such as GPT-3, despite the significant difference in size, and that integrating large language models into our framework improves their performance.
BibTeX
@misc{nobari2023dtt,
  author = {Arash Dargahi Nobari and Davood Rafiei},
  title = {{DTT: An Example-Driven Tabular Transformer by Leveraging Large Language Models}},
  year = {2023},
  eprint = {2303.06748},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.06748},
}
Figure