Attend, Memorize and Generate: Towards Faithful Table-to-Text Generation in Few Shots

Wenting Zhao, Ye Liu, Yao Wan, Philip S. Yu

Abstract

Faithful table-to-text generation remains a significant challenge, particularly in few-shot scenarios where models often hallucinate information or omit key details. This paper introduces the Attend, Memorize and Generate (AMG) framework, which leverages a dual-layer attention mechanism and a dynamic memory unit to track described table slots. By treating table attributes as distinct entities, AMG ensures high factual accuracy and fluency even with minimal training data.

The Problem: Hallucination and Structural Loss

Table-to-text generation involves converting semi-structured data, such as Wikipedia infoboxes, into natural language descriptions. While large pre-trained models like GPT-2 and BART are highly fluent, they frequently struggle with faithfulness—the ability to strictly adhere to provided facts. In few-shot settings, these models frequently suffer from "generation bias," where they rely on pre-training knowledge to fill gaps, leading to hallucinations (e.g., assigning incorrect nationalities or names). Furthermore, standard models often flatten tables into simple text strings, losing the inherent relationships between attributes and their values.

The AMG Framework

The proposed AMG framework, built on the UniLM architecture, introduces three key innovations to maintain grounding during generation:

  1. Multi-Granularity Attention: Standard models process text token-by-token. AMG adds a second layer of attention at the "table slot" level, forcing the model to recognize multi-word values (e.g., "August 10, 1988") as single units belonging to specific slots like birth_date.
  2. Dynamic Memory Unit: Acting as a digital checklist, this unit tracks which table slots have been "visited" during generation. Updating the memory state for a slot like name encourages the model to move on to other facts, preventing repetition or omission.
  3. Two-Phase Task-Adaptive Pre-training: To effectively initialize these components for few-shot tasks, the authors use large-scale unlabeled table data to "warm up" the slot attention and memory mechanisms before final fine-tuning.
The AMG architecture consists of three integrated phases: (a) Multi-granularity attention over table slots and history, (b) a dynamic memory unit to track slot coverage, and (c) token generation based on the updated memory state.

Experimental Results

The researchers evaluated AMG across three domains from the Wiki dataset: Humans (biographies), Books (literary works), and Songs (musical tracks). Testing was conducted in extreme few-shot settings, using as few as 50 to 500 training examples.

Performance comparison of AMG against state-of-the-art baselines across three domains. AMG consistently achieves higher scores in faithfulness (PARENT) and fluency metrics (BLEU-4, ROUGE-L).

Key Findings

  • Superior Faithfulness: AMG consistently outperformed BART on the PARENT metric, a specialized score for table-to-text faithfulness. In the Humans domain with 200 training examples, AMG showed a 2.72% improvement over the next best model.
  • Fluency Preservation: Despite added structural constraints, AMG maintained or improved traditional scores like BLEU-4 and ROUGE-L, proving that factual accuracy does not require sacrificing natural-sounding prose.
  • Human Evaluation: Blind tests showed that human annotators ranked AMG's outputs as more informative and higher quality than those from BART, particularly in the 50-shot and 100-shot categories.

Conclusion

The AMG framework demonstrates that explicitly modeling table structure and tracking generation state are vital for faithful table-to-text generation. By moving beyond simple sequence-to-sequence modeling, AMG provides a more reliable solution for generating accurate descriptions from structured data in low-resource environments.

Summary updated

BibTeX
@inproceedings{zhao2021attend,
  author = {Wenting Zhao and Ye Liu and Yao Wan and Philip S. Yu},
  title = {{Attend, Memorize and Generate: Towards Faithful Table-to-Text Generation in Few Shots}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.findings-emnlp.347},
}
Figure