TableNet: A Large-Scale Table Dataset with LLM-Powered Autonomous Generation

Ruilin Zhang, Kai YangAAAI 2026 · OpenReview

TL;DR

This paper introduces TableNet, a massive new dataset for Table Structure Recognition (TSR), generated by a novel LLM-powered autonomous system. This system can create diverse, realistic tables with fine-grained control over their visual and structural properties, addressing the limitations of existing datasets and enabling more robust TSR models.

What Problem Does This Paper Address?

Table Structure Recognition (TSR) is crucial for understanding tabular data, but it's a challenging task due to the wide variety of table layouts, styles, and semantic complexities found in real-world documents. Existing datasets for TSR are often limited in scale, diversity, and annotation quality, which hinders the ability of models, especially Large Language Models (LLMs), to fully leverage their logical reasoning capabilities for accurate table parsing. This gap makes it difficult to train models that can generalize well to unseen, complex tables.

Methods

The core innovation is an LLM-powered autonomous multi-agent system for generating tables. This system breaks down table creation into several stages:

  • Schema Planning: An LLM determines table dimensions, layout (including row/column spans), and visual styling based on user input.
  • Content Generation: Specialized LLMs generate table topics and then fill in the header and body cells with semantically relevant content.
  • Augmentation: A contrastive learning-inspired strategy is used to create variations of tables by copying, deleting, swapping, or altering elements, increasing diversity.
  • Validation: A mixed strategy involving heuristics and LLM ranking checks for structural correctness, topic relevance, and semantic consistency.

For the TSR recognition part, the authors employ a diversity-based active learning paradigm. This strategy intelligently selects the most informative samples from a pool of generated and real-world tables to fine-tune the TSR model, aiming to achieve high performance with fewer training samples.

Datasets

The paper introduces TableNet, a large-scale dataset comprising:

  • Agent-generated tables: Synthetically created using the LLM-powered multi-agent system, offering extensive control over diversity.
  • Crawled real-world tables: Collected from telecommunication-related PDF and Word documents.
  • Augmented open-source datasets: Existing structured table data rendered into images.

TableNet aims to provide a more comprehensive and nuanced dataset than existing benchmarks like PubTabNet, FinTabNet, and TableBank, which are noted for their limitations in diversity and scale.

Key Insights and Ideas

  • LLMs as Generative Engines for Datasets: The paper demonstrates that LLMs can be effectively orchestrated into multi-agent systems to autonomously generate large-scale, diverse, and well-annotated datasets, overcoming the limitations of manual collection.
  • Controllable Generation is Key: By integrating parameters for visual style, structure, and semantics, the system allows for theoretically infinite, domain-agnostic, and style-flexible table generation, which is crucial for creating robust TSR models.
  • Active Learning for Diverse Tables: The diversity of TableNet makes it an ideal candidate for active learning strategies, which can efficiently train models by focusing on the most informative examples, especially for complex table structures.
  • Bridging the Gap Between Synthetic and Real-World Data: The combination of synthetic generation and real-world crawling in TableNet aims to create a dataset that better prepares models for the complexities of real-world tabular data.

Key Results

  • The TableNet dataset contains 445K tables, significantly larger than many existing TSR datasets.
  • The LLM-powered generation system can create tables with controllable visual styles, structural complexity (including spans), and semantic domains.
  • Models fine-tuned on TableNet using active learning achieved competitive performance on the TableNet test set while requiring substantially fewer training samples compared to baselines.
  • Models trained on TableNet outperformed models trained on other public datasets when evaluated on unseen real-world tables, demonstrating superior generalizability. For instance, a model trained on TableNet achieved a TEDS of 0.7403 on unseen real-world tables, significantly better than models trained on other datasets.

Limitations and Future Work

  • LLM Pretraining Limitations: The diversity of generated content is still bounded by the underlying LLM's pretraining distribution and reasoning capabilities, potentially missing edge cases or highly domain-specific formats.
  • Content Quality Variability: The quality of generated table images can vary due to the LLM's knowledge of user-specified domains and languages, potentially leading to irrelevant content or semantic inconsistencies.
  • Future Work: The authors suggest that integrating a powerful generation agent with TableNet will significantly advance TSR research. Further exploration of LLM capabilities for TSR and dataset generation is implied.

Summary updated

Abstract

Table Structure Recognition (TSR) requires the logical reasoning ability of large language models (LLMs) to handle complex table layouts, but current datasets are limited in scale and quality, hindering effective use of this reasoning capacity. We thus present \textbf{TableNet dataset}, a new table structure recognition dataset collected and generated through multiple sources. Central to our approach is the first \textbf{LLM-powered autonomous table generation and recognition multi-agent system} that we developed. The generation part of our system integrates controllable visual, structural, and semantic parameters into the synthesis of table images. It facilitates the creation of a wide array of semantically coherent tables, adaptable to user-defined configurations along with annotations, thereby supporting large-scale and detailed dataset construction. This capability enables a comprehensive and nuanced table image annotation taxonomy, potentially advancing research in table-related domains. In contrast to traditional data collection methods, This approach facilitates the theoretically infinite, domain-agnostic, and style-flexible generation of table images, ensuring both efficiency and precision. The recognition part of our system is a diversity-based active learning paradigm that utilizes tables from multiple sources and selectively samples most informative data to finetune a model, achieving a competitive performance on TableNet test set while reducing training samples by a large margin compared with baselines, and a much higher performance on web-crawled real-world tables compared with models trained on predominant table datasets. To the best of our knowledge, this is the first work which employs the concept of active learning into the structure recognition of tables which is diverse in numbers of rows or columns, merged cells, cell contents, etc, which fits better for diversity-based active learning.
BibTeX
@inproceedings{zhang2026tablenet,
  author = {Ruilin Zhang and Kai Yang},
  title = {{TableNet: A Large-Scale Table Dataset with LLM-Powered Autonomous Generation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  url = {https://openreview.net/forum?id=QjHpq1QO03},
}
Figure