Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation

Peng Wang, Junyang Lin, An Yang, Chang Zhou, Yichang Zhang, Jingren Zhou, Hongxia Yang

Abstract

Table-to-text generation aims to produce descriptive natural language from structured data. While traditional autoregressive models are fluent, they often suffer from hallucinations and low information coverage. We present SANA (Skeleton-based Autoregressive and Non-Autoregressive generation), a two-stage framework that ensures faithfulness by first constructing a factual "skeleton" and then refining it through iterative non-autoregressive editing. SANA achieves state-of-the-art results, including a near-perfect PARENT-T recall of 99.47 on the WikiPerson dataset.

The Challenge of Faithful Generation

Table-to-text generation is the task of converting structured data—typically key-value pairs—into fluent, natural language descriptions. For example, given a table about an actress, a model should generate a biography that includes her name, birth date, and occupation without inventing new details.

Example of a source table for Thaila Ayala, providing the structured data used for generation.

Standard end-to-end autoregressive models (like standard Seq2Seq Transformers) struggle with two major flaws:

  1. Hallucination: Generating fluent text that invents facts not present in the source table.
  2. Low Coverage: Omitting key attributes provided in the table.

While previous two-stage models attempted to use "content plans," they often treated these plans as mere suggestions, allowing the decoder to still hallucinate or ignore the plan during the final generation phase.

The SANA Framework

SANA introduces a novel two-stage architecture that fundamentally changes how the final text is decoded by combining the strengths of autoregressive and non-autoregressive (NAR) models.

The SANA architecture: Stage 1 (left) uses a pointer network to construct a factual skeleton; Stage 2 (right) performs iterative editing via insertion and deletion.

Stage 1: Skeleton Construction (Autoregressive)

The model first uses a Transformer-based pointer network to select salient tokens directly from the source table. These tokens are arranged into a "skeleton"—a barebones sequence of essential facts (e.g., "Thaila Ayala born April 14 1986 Brazil actress model").

Stage 2: Surface Realization (Non-Autoregressive)

Instead of generating the final sentence word-by-word, SANA uses an edit-based NAR Transformer. It takes the skeleton as its initial input and iteratively refines it using simultaneous insertion and deletion operations.

The Key Innovation: SANA applies hard constraints during this stage. It masks the deletion classifier for tokens belonging to the skeleton, forbidding the model from removing them. This guarantees that selected facts survive into the final text, while the NAR model focuses purely on adding grammatical "glue" to ensure fluency without hallucination.

Evaluation and Results

The authors evaluated SANA on the WikiBio and WikiPerson datasets. WikiPerson is a particularly clean dataset designed to test strict faithfulness, while WikiBio is larger but noisier.

  • Near-Perfect Coverage: On WikiPerson, SANA achieved a PARENT-T recall of 99.47, improving over the previous best result by more than 10 points. The PARENT metric is crucial here as it evaluates outputs against both the reference text and the source table.
  • Human Evaluation: In blind tests, SANA ranked highest in Fluency (3.46), Coverage (3.72), and Correctness (3.11) on a scale of 0 to 4.
  • Robustness: Even on the noisy WikiBio dataset, SANA outperformed previous models, achieving a BLEU score of 45.78.

Conclusion

SANA demonstrates that treating content plans as hard constraints rather than soft prompts is vital for faithful table-to-text generation. By combining a factual "sketch" with iterative non-autoregressive "refinement," the model effectively eliminates the common pitfalls of hallucination and information loss, setting a new standard for informative and reliable data-to-text systems.

Summary updated

Abstract

Table-to-text generation refers to generating a descriptive text from a key-value table. Traditional autoregressive methods, though can generate text with high fluency, suffer from low coverage and poor faithfulness problems. To mitigate these problems, we propose a novel Skeleton-based two-stage method that combines both Autoregressive and Non-Autoregressive generations (SANA). Our approach includes: (1) skeleton generation with an autoregressive pointer network to select key tokens from the source table; (2) edit-based non-autoregressive generation model to produce texts via iterative insertion and deletion operations. By integrating hard constraints from the skeleton, the non-autoregressive model improves the generation's coverage over the source table and thus enhances its faithfulness. We conduct automatic and human evaluations on both WikiPerson and WikiBio datasets. Experimental results demonstrate that our method outperforms the previous state-of-the-art methods in both automatic and human evaluation, especially on coverage and faithfulness. In particular, we achieve PARENT-T recall of 99.47 in WikiPerson, improving over the existing best results by more than 10 points.
BibTeX
@inproceedings{wang2021sketch,
  author = {Peng Wang and Junyang Lin and An Yang and Chang Zhou and Yichang Zhang and Jingren Zhou and Hongxia Yang},
  title = {{Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=tTHuXSwF4C7},
}
Figure