Bi-Directional Multi-Granularity Generation Framework for Knowledge Graph-to-Text with Large Language Model

Haowei Du, Chen Li, Dinghao Zhang, Dongyan Zhao

Introduction

Knowledge Graph-to-Text (KG-to-text) generation is the process of transforming structured data, such as triples (subject, relation, object), into coherent natural language descriptions. While Large Language Models (LLMs) have significantly advanced this field, they often struggle as the input graph grows in complexity.

Standard models typically attempt to ingest an entire graph and generate a multi-sentence paragraph in a single pass. This often leads to "hallucinations" or "relation mixing," where the model incorrectly attributes a fact from one part of the graph to an entity in another. For example, as illustrated in the figure below, a model might incorrectly state that a university has a "monarch" simply because that relation appeared elsewhere in the graph context.

Comparison of correct vs. incorrect KG-to-text generation showing the problem of relation mixing.

The BDMG Framework

To mitigate these issues, the authors propose the Bi-Directional Multi-Granularity (BDMG) framework. This approach operates on two main axes: a "divide-and-conquer" forward generation strategy and a backward verification task.

Overview of the BDMG framework featuring forward sentence-level generation and backward relation extraction.

Forward Sentence-Level Generation

Instead of generating a whole paragraph at once, the model follows a sequential, three-step "Chain of Thought" style process:

  1. Selection: The model identifies a specific subset of triples from the Knowledge Graph (KG) needed for the current sentence.
  2. Generation: It writes a single, semantically complete sentence based strictly on those triples.
  3. Aggregation: This process repeats until all triples are covered, at which point the sentences are aggregated into a final, polished graph-level text.

Backward Relation Extraction (RE)

To ensure the model truly understands the relationships it describes, the framework introduces a joint training task. The model is presented with its own generated text and asked to infer the original relation between two entities. This "backward" check acts as a powerful regularizer, incentivizing the model to encode relational information more accurately during the initial generation phase.

Experimental Results

The researchers evaluated BDMG on the WebNLG benchmark (version 2.0, Constrained split). The framework was implemented using Flan T5 (3B and 11B versions) as the backbone, utilizing LoRA (Low-Rank Adaptation) for efficient fine-tuning.

Key findings include:

  • State-of-the-Art Performance: BDMG-11B achieved a 70.65 BLEU score, outperforming the previous best model (Plan Selection) by approximately 8.5 points.
  • Metric Consistency: The model showed significant gains across multiple metrics, including METEOR (50.30) and ROUGE (81.36), indicating improvements in both accuracy and fluency.
  • Ablation Success: Removing the sequential generation caused a drop of 0.7 BLEU, while removing the backward RE task caused a drop of 0.3 BLEU, proving that both components are essential to the framework's success.

Conclusion

The BDMG framework demonstrates that breaking down graph-to-text generation into sentence-level granularity significantly reduces noise and hallucinations. By mirroring the human writing process—grouping related facts into distinct sentences and verifying them—the model achieves a new level of reliability in structured data verbalization. Future work will focus on scaling this approach to massive, web-scale knowledge graphs with hundreds of interconnected facts.

Summary updated

Abstract

The knowledge graph-to-text (KG-to-text) generation task aims to synthesize coherent and engaging sentences that accurately convey the complex information derived from an input knowledge graph. Existing methods generate the whole target text based on all KG triples at once and may incorporate incorrect KG triples for each sentence. To this end, we propose the bi-directional multi-granularity generation framework. Instead of generating the whole text at a time, we construct the sentence level generation based on the corresponding triples and generate the graph-level text as a result. Moreover, we design a backward relation extraction task to enhance the correctness of relational information. Our method achieves the new state-of-the-art in benchmark dataset WebNLG and further analysis shows the efficiency of different modules.
BibTeX
@inproceedings{du2024bi,
  author = {Haowei Du and Chen Li and Dinghao Zhang and Dongyan Zhao},
  title = {{Bi-Directional Multi-Granularity Generation Framework for Knowledge Graph-to-Text with Large Language Model}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.acl-short.14},
}
Figure