Informative Text Generation from Knowledge Triples

Zihao Fu, Yijiang River Dong, Lidong Bing, Wai LamarXiv 2022 · DOI · arXiv

Presented at EMNLP 2022, this research addresses a significant bottleneck in Knowledge-Base-to-text (KB-to-text) generation: the trade-off between strict faithfulness and human-like informativeness. While traditional models are optimized to avoid "hallucinations," they often produce rigid, unhelpful text. This paper introduces the task of Informative Text Generation (ITG) and a novel architecture to solve it.

The Informative Text Generation Task

Standard KB-to-text tasks assume a one-to-one mapping between input data and output text. For instance, given the triple <Bill Gates, FounderOf, Microsoft>, a typical model outputs "Bill Gates founded Microsoft." However, human narratives naturally incorporate relevant background context—such as a person's birthplace or a company's headquarters—to provide a more comprehensive summary.

Conceptual overview of Informative Text Generation (ITG), where input triples are augmented with auxiliary knowledge from memory to produce richer narratives.

ITG requires models to "read between the lines" by generating text that includes relevant background information not explicitly provided in the input knowledge triples. This shift moves generation from mere translation of data to the creation of informative, human-like summaries.

The iMAG Framework

To tackle this task, the authors propose the information Memory Augmented Generator (iMAG). The architecture is designed to store and retrieve auxiliary knowledge during both training and inference.

Architecture and Training

The iMAG model consists of two primary components:

  1. Information Memory: A learnable memory matrix ($V$) that stores extra information encountered during training. To ensure the model actually uses this memory, the researchers employed a "triple-dropping" strategy: randomly removing input triples during training. This forces the model to retrieve the missing facts from its internal memory using the remaining entities as a query.
  2. Sequence-to-Sequence Network: An LSTM-based encoder-decoder with attention and a copy mechanism. It integrates the original input triples with retrieved memory vectors to generate the final narrative.
The iMAG architecture, featuring the memory matrix (V), keys (K), and the Repeating Sentence Penalty (RSP) integrated into the loss function.

Handling Repetition

A common challenge in memory-augmented generation is content repetition. To mitigate this, the authors introduced the Repeating Sentence Penalty (RSP). By calculating the Longest Repeating Non-overlapping Substring, the model is penalized during training for generating redundant information, ensuring the output remains concise while being rich in content.

Key Results and Insights

The researchers evaluated iMAG on an adapted version of the WebNLG v2 dataset. Key findings include:

  • Enhanced Richness: In human evaluations, iMAG+RSP scored significantly higher in "Richness" (7.02) compared to a standard Seq2Seq baseline (5.84).
  • Substantial Output: The model generated significantly longer narratives, averaging 35.4 words per sample, compared to the baseline's 23.7 words.
  • Zero-Triple Generation: Remarkably, iMAG demonstrated the ability to generate coherent entity biographies when provided with only an entity name and zero input triples, relying entirely on its internal "common sense" memory.
  • Repetition Control: The RSP mechanism successfully reduced the repetition ratio (LRNSR) from 0.120 to 0.102 without compromising the quality of the generated information.

Conclusion

The iMAG framework represents a shift toward more functional and informative AI text generation. By treating memory as a repository for domain-specific common sense, the model moves beyond strict data-to-text mapping. While challenges remain in balancing plausibility with richness and scaling to open-domain settings, this work provides a robust foundation for generating narratives that are as helpful as they are faithful.

Summary updated

Abstract

As the development of the encoder-decoder architecture, researchers are able to study the text generation tasks with broader types of data. Among them, KB-to-text aims at converting a set of knowledge triples into human readable sentences. In the original setting, the task assumes that the input triples and the text are exactly aligned in the perspective of the embodied knowledge/information. In this paper, we extend this setting and explore how to facilitate the trained model to generate more informative text, namely, containing more information about the triple entities but not conveyed by the input triples. To solve this problem, we propose a novel memory augmented generator that employs a memory network to memorize the useful knowledge learned during the training and utilizes such information together with the input triples to generate text in the operational or testing phase. We derive a dataset from WebNLG for our new setting and conduct extensive experiments to investigate the effectiveness of our model as well as uncover the intrinsic characteristics of the setting.
BibTeX
@misc{fu2022informative,
  author = {Zihao Fu and Yijiang River Dong and Lidong Bing and Wai Lam},
  title = {{Informative Text Generation from Knowledge Triples}},
  year = {2022},
  eprint = {2209.12733},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2209.12733},
}
Figure