WikiTableT: A Large-Scale Data-to-Text Dataset for Generating Wikipedia Article Sections

Mingda Chen, Sam Wiseman, Kevin Gimpel

Abstract

Datasets for data-to-text generation typically focus either on multi-domain, single-sentence generation or on single-domain, long-form generation. In this work, we cast generating Wikipedia sections as a data-to-text generation task and create a large-scale dataset, WikiTableT, that pairs Wikipedia sections with their corresponding tabular data and various metadata. WikiTableT contains millions of instances, covering a broad range of topics, as well as a variety of flavors of generation tasks with different levels of flexibility. We benchmark several training and decoding strategies on WikiTableT. Our qualitative analysis shows that the best approaches can generate fluent and high quality texts but they struggle with coherence and factuality, showing the potential for our dataset to inspire future work on long-form generation.

Introduction

Data-to-text generation—the task of transforming structured information into natural language—has traditionally been bifurcated. Research has either focused on multi-domain, single-sentence generation (e.g., describing a single table row) or single-domain, long-form generation (e.g., sports game summaries). A significant gap exists in multi-domain, long-form generation, where models must synthesize diverse data into coherent, multi-sentence narratives across various topics.

The authors address this by introducing WikiTableT, a dataset that frames the creation of Wikipedia sections as a data-to-text task. This requires models to handle the complexity of Wikipedia's diverse subject matter while maintaining the structure and factual density of a full article section.

The WikiTableT Dataset

WikiTableT is a massive benchmark designed to push the boundaries of long-form generation. It comprises approximately 1.5 million instances, a significant increase over previous datasets like RotoWire (~5k instances). The dataset covers 147,000 unique record types across a vast array of Wikipedia topics, including biographies, films, and geographical settlements.

Example of structured data input for a Wikipedia section, including attributes like film genre, director, and starring cast.

The data is sourced from a combination of "Article Data" (infoboxes and Wikidata) and "Section Data" (hyperlinks and entities specific to a section). This multi-level data structure allows for different flavors of generation tasks with varying levels of flexibility and open-endedness.

Methodology

The researchers evaluated several Transformer-based architectures using specialized training and decoding strategies:

  • Table Linearization: Tables are "flattened" into sequences using special tokens (e.g., <boc>, <bov>) and subscript indices to maintain alignment between attributes and values.
  • Training Enhancements: Strategies included alpha-entmax for sparsity, a Copy Mechanism to directly transfer entities from the table to the text, and a Cyclic Loss function that penalizes the model if the original table cannot be reconstructed from the generated text.
  • Decoding Strategies: The study compared greedy decoding and beam search against Nucleus Sampling and n-gram blocking to mitigate common issues like word repetition.

Key Results and Insights

The experimental results highlight the challenges of long-form generation. The best-performing configuration utilized a Large model with a copy mechanism and cyclic loss, combined with beam search and n-gram blocking.

Performance metrics for various training and decoding strategies on WikiTableT, showing BLEU and PARENT scores.

Key findings include:

  • Repetition Management: "Hard" constraints like n-gram blocking were significantly more effective at reducing repetition than "soft" learned strategies.
  • Fluency vs. Factuality: While models achieved high fluency scores (often rivaling human-written text in grammar and coherence), they frequently suffered from "hallucinations" or logical contradictions relative to the input data.
  • Contextual Importance: Models performed markedly better when provided with section-specific data rather than general article-level infoboxes, emphasizing the need for local context in document generation.

Conclusion

WikiTableT provides a rigorous benchmark for multi-domain, long-form data-to-text generation. While modern models can produce fluent prose, ensuring long-term coherence and strict factuality remains a major hurdle. Future work may benefit from incorporating large-scale pre-trained models or explicit fact-checking modules to bridge the gap between sounding natural and being factually accurate.

Summary updated

Abstract

Datasets for data-to-text generation typically focus either on multi-domain, single-sentence generation or on single-domain, long-form generation. In this work, we cast generating Wikipedia sections as a data-to-text generation task and create a large-scale dataset, WikiTableT, that pairs Wikipedia sections with their corresponding tabular data and various metadata. WikiTableT contains millions of instances, covering a broad range of topics, as well as a variety of flavors of generation tasks with different levels of flexibility. We benchmark several training and decoding strategies on WikiTableT. Our qualitative analysis shows that the best approaches can generate fluent and high quality texts but they struggle with coherence and factuality, showing the potential for our dataset to inspire future work on long-form generation.
BibTeX
@inproceedings{chen2021wikitablet,
  author = {Mingda Chen and Sam Wiseman and Kevin Gimpel},
  title = {{WikiTableT: A Large-Scale Data-to-Text Dataset for Generating Wikipedia Article Sections}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.17},
}
Figure