MuCAL: Contrastive Alignment for Preference-Driven KG-to-Text Generation

Yifei Song, Claire GardentEMNLP 2025

TL;DR

This paper introduces MuCAL, a novel framework for generating natural language text from knowledge graphs (KGs) by leveraging preference learning. It tackles the challenge of scarce reliable preference data by automatically creating it using a KG-text alignment model and then fine-tuning large language models (LLMs) with Direct Preference Optimization (DPO), leading to improved factual accuracy and out-of-domain generalization.

What Problem Does This Paper Address?

Generating coherent and factually accurate text from knowledge graphs (KGs) is a challenging task. While existing methods often rely on large, manually curated datasets, creating such data is expensive and time-consuming. Furthermore, fine-tuned models struggle to generalize to unseen KGs with novel entities or properties. Preference learning offers a promising avenue for improving generation quality by aligning models with human preferences, but it requires reliable preference data, which is particularly scarce for KG-to-text generation. This paper addresses the need for a KG-to-text generation approach that can learn from preferences without requiring manual annotation and can generalize well to unseen data.

Methods

The core of the proposed solution is MuCAL (Multilingual Contrastive Alignment Learning). MuCAL is a multilingual KG/Text alignment model trained using contrastive learning. It learns to map KGs and texts into a shared semantic space, enabling robust cross-modal retrieval across multiple languages. This alignment model is then used to automatically construct preference data.

The process involves:

  1. Automated Preference Data Construction:
    • Diverse LLMs generate multiple candidate texts for a given KG.
  2. MuCAL's KG/Text similarity scores are used to rank these candidate texts.
  3. Preference triplets (graph, preferred text, rejected text) are formed by selecting the highest and lowest ranked texts. This bypasses the need for human annotation.
  4. Direct Preference Optimization (DPO):
    • The automatically generated preference data is used to fine-tune LLMs using DPO.
  5. DPO directly optimizes the generation policy based on these preferences, aligning the model's output with graph semantics without explicit reward modeling.

The paper also explores both bi-encoder and cross-encoder architectures for the MuCAL alignment model, with the bi-encoder showing better performance when incorporating "hard negatives" (specifically corrupted graphs) during training.

Datasets

The paper utilizes and creates several datasets:

  • WebNLG: A dataset of manually aligned KGs and English textual descriptions. Used for training and testing.
  • KELM: A large-scale dataset of KG/English text pairs extracted from Wikidata and Wikipedia using distant supervision. Two subsets are used:
  • KELM-Q1: A filtered version for training.
  • KELM-TEST: A manually validated subset for in-domain evaluation.
  • MULTI-TRAIN-SILVER: English texts from WebNLG and KELM are machine-translated into five target languages (Arabic, Chinese, French, Russian, Spanish) to create a multilingual training set.
  • Test Sets:
    • MULTI-TEST-1K: A balanced subset of KG/English text pairs from WebNLG and KELM, translated into five languages, for retrieval evaluation.
  • MULTI-WEBNLG-TEST: WebNLG test set graphs with their translated verbalizations.
  • MULTI-TEST-1K-CORR: An extension of Multi-Test-1K with corrupted graphs to evaluate robustness.
  • GOLD-OOD-472: A newly created out-of-domain dataset with unseen graphs and manually edited English reference texts, designed to assess generalization.

Key Insights and Ideas

  • Automated Preference Data is Key: The ability to automatically generate high-quality preference data using a KG-text alignment model is crucial for enabling preference learning in KG-to-text generation without human annotation.
  • MuCAL as a Better Signal for DPO: The KG/Text similarity scores from MuCAL provide a more effective signal for DPO training compared to existing reference-less metrics, leading to better factual accuracy and generalization.
  • Hard Negatives Improve Robustness: Incorporating "hard negatives" (specifically corrupted graphs like swapped entities or predicates) during the training of the KG/Text alignment model significantly improves its ability to distinguish correct graphs from corrupted ones, which is vital for robustness.
  • Bi-Encoder Outperforms Cross-Encoder for Out-of-Domain: The bi-encoder architecture, especially when trained with hard negatives, shows better generalization on unseen (out-of-domain) data compared to the cross-encoder.
  • DPO Supports Paraphrasing: DPO-tuned models tend to paraphrase rather than memorize references, suggesting a better ability to generalize and produce more natural-sounding text, especially on unseen data.

Key Results

  • Improved Retrieval: MuCAL models outperform baselines on retrieval tasks across multiple languages and test sets, especially on challenging datasets like Multi-Test-1K-Corr.
  • Superior DPO Performance: DPO models trained with MuCAL-generated preferences achieve significantly better out-of-domain generalization (e.g., on GOLD-OOD-472) compared to standard instruction tuning and few-shot prompting. For instance, DPO-HARD-MuCAL shows notably stronger generalization on unseen data than instruction-tuned Qwen.
  • Better KG/Text Similarity Signal: MuCAL-based KG/Text similarity metrics provide a better signal for DPO than existing metrics like EREDAT, FactSpotter, and Data-QuestEval, leading to better performance across all test sets.
  • Cost-Effective Pipeline: The automated preference construction pipeline is cost-effective, incurring minimal costs even for large datasets.

Limitations and Future Work

  • English-Centric DPO: Current DPO experiments focus solely on English KG-to-Text generation. Extending the pipeline to other languages for DPO training is a key future direction.
  • LLM Dependency: The approach relies on high-quality texts generated by large LLMs, which can be costly and potentially restrictive.
  • Multilingual DPO Efficacy: While MuCAL is trained on multilingual data, its effectiveness in guiding non-English DPO training remains to be demonstrated.
  • Future Work: The authors plan to extend their approach to multilingual DPO-trained KG-to-Text generation and explore alternative preference learning algorithms for KG-to-Text generation to enhance fidelity and adaptability across diverse domains and languages.

Summary updated

BibTeX
@inproceedings{song2025mucal,
  author = {Yifei Song and Claire Gardent},
  title = {{MuCAL: Contrastive Alignment for Preference-Driven KG-to-Text Generation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure