Abstract
Faithful table-to-text generation remains a significant challenge, particularly in few-shot scenarios where models often hallucinate information or omit key details. This paper introduces the Attend, Memorize and Generate (AMG) framework, which leverages a dual-layer attention mechanism and a dynamic memory unit to track described table slots. By treating table attributes as distinct entities, AMG ensures high factual accuracy and fluency even with minimal training data.
The Problem: Hallucination and Structural Loss
Table-to-text generation involves converting semi-structured data, such as Wikipedia infoboxes, into natural language descriptions. While large pre-trained models like GPT-2 and BART are highly fluent, they frequently struggle with faithfulness—the ability to strictly adhere to provided facts. In few-shot settings, these models frequently suffer from "generation bias," where they rely on pre-training knowledge to fill gaps, leading to hallucinations (e.g., assigning incorrect nationalities or names). Furthermore, standard models often flatten tables into simple text strings, losing the inherent relationships between attributes and their values.
The AMG Framework
The proposed AMG framework, built on the UniLM architecture, introduces three key innovations to maintain grounding during generation:
- Multi-Granularity Attention: Standard models process text token-by-token. AMG adds a second layer of attention at the "table slot" level, forcing the model to recognize multi-word values (e.g., "August 10, 1988") as single units belonging to specific slots like
birth_date. - Dynamic Memory Unit: Acting as a digital checklist, this unit tracks which table slots have been "visited" during generation. Updating the memory state for a slot like
nameencourages the model to move on to other facts, preventing repetition or omission. - Two-Phase Task-Adaptive Pre-training: To effectively initialize these components for few-shot tasks, the authors use large-scale unlabeled table data to "warm up" the slot attention and memory mechanisms before final fine-tuning.
Experimental Results
The researchers evaluated AMG across three domains from the Wiki dataset: Humans (biographies), Books (literary works), and Songs (musical tracks). Testing was conducted in extreme few-shot settings, using as few as 50 to 500 training examples.
Key Findings
- Superior Faithfulness: AMG consistently outperformed BART on the PARENT metric, a specialized score for table-to-text faithfulness. In the Humans domain with 200 training examples, AMG showed a 2.72% improvement over the next best model.
- Fluency Preservation: Despite added structural constraints, AMG maintained or improved traditional scores like BLEU-4 and ROUGE-L, proving that factual accuracy does not require sacrificing natural-sounding prose.
- Human Evaluation: Blind tests showed that human annotators ranked AMG's outputs as more informative and higher quality than those from BART, particularly in the 50-shot and 100-shot categories.
Conclusion
The AMG framework demonstrates that explicitly modeling table structure and tracking generation state are vital for faithful table-to-text generation. By moving beyond simple sequence-to-sequence modeling, AMG provides a more reliable solution for generating accurate descriptions from structured data in low-resource environments.
Summary updated