Abstract
Table-to-text generation aims to produce descriptive natural language from structured data. While traditional autoregressive models are fluent, they often suffer from hallucinations and low information coverage. We present SANA (Skeleton-based Autoregressive and Non-Autoregressive generation), a two-stage framework that ensures faithfulness by first constructing a factual "skeleton" and then refining it through iterative non-autoregressive editing. SANA achieves state-of-the-art results, including a near-perfect PARENT-T recall of 99.47 on the WikiPerson dataset.
The Challenge of Faithful Generation
Table-to-text generation is the task of converting structured data—typically key-value pairs—into fluent, natural language descriptions. For example, given a table about an actress, a model should generate a biography that includes her name, birth date, and occupation without inventing new details.
Standard end-to-end autoregressive models (like standard Seq2Seq Transformers) struggle with two major flaws:
- Hallucination: Generating fluent text that invents facts not present in the source table.
- Low Coverage: Omitting key attributes provided in the table.
While previous two-stage models attempted to use "content plans," they often treated these plans as mere suggestions, allowing the decoder to still hallucinate or ignore the plan during the final generation phase.
The SANA Framework
SANA introduces a novel two-stage architecture that fundamentally changes how the final text is decoded by combining the strengths of autoregressive and non-autoregressive (NAR) models.
Stage 1: Skeleton Construction (Autoregressive)
The model first uses a Transformer-based pointer network to select salient tokens directly from the source table. These tokens are arranged into a "skeleton"—a barebones sequence of essential facts (e.g., "Thaila Ayala born April 14 1986 Brazil actress model").
Stage 2: Surface Realization (Non-Autoregressive)
Instead of generating the final sentence word-by-word, SANA uses an edit-based NAR Transformer. It takes the skeleton as its initial input and iteratively refines it using simultaneous insertion and deletion operations.
The Key Innovation: SANA applies hard constraints during this stage. It masks the deletion classifier for tokens belonging to the skeleton, forbidding the model from removing them. This guarantees that selected facts survive into the final text, while the NAR model focuses purely on adding grammatical "glue" to ensure fluency without hallucination.
Evaluation and Results
The authors evaluated SANA on the WikiBio and WikiPerson datasets. WikiPerson is a particularly clean dataset designed to test strict faithfulness, while WikiBio is larger but noisier.
- Near-Perfect Coverage: On WikiPerson, SANA achieved a PARENT-T recall of 99.47, improving over the previous best result by more than 10 points. The PARENT metric is crucial here as it evaluates outputs against both the reference text and the source table.
- Human Evaluation: In blind tests, SANA ranked highest in Fluency (3.46), Coverage (3.72), and Correctness (3.11) on a scale of 0 to 4.
- Robustness: Even on the noisy WikiBio dataset, SANA outperformed previous models, achieving a BLEU score of 45.78.
Conclusion
SANA demonstrates that treating content plans as hard constraints rather than soft prompts is vital for faithful table-to-text generation. By combining a factual "sketch" with iterative non-autoregressive "refinement," the model effectively eliminates the common pitfalls of hallucination and information loss, setting a new standard for informative and reliable data-to-text systems.
Summary updated