FusionAdapter for Few-Shot Relation Learning in Multimodal Knowledge Graphs

Ran Liu, Yuan Fang, Xiaoli LiarXiv 2025 · DOI · arXiv

Knowledge Graphs (KGs) serve as the foundation for modern search and recommendation systems, yet they are frequently incomplete. Multimodal Knowledge Graphs (MMKGs) attempt to address these gaps by augmenting traditional symbolic triples—head, relation, and tail—with auxiliary data such as images and text. However, MMKGs face a significant hurdle in Few-Shot Relation Learning (FSRL): the ability to learn a brand-new relation from only a handful of examples.

In the paper "FusionAdapter for Few-Shot Relation Learning in Multimodal Knowledge Graphs," researchers introduce a novel framework designed to integrate diverse data streams without sacrificing the unique characteristics of each modality.

The Multimodal Challenge: Over-Homogenization

Traditional KGs represent facts as symbolic triples, such as (Wichita Falls, Capital_of, Wichita County). MMKGs enrich these facts with visual and textual context.

An example of an MMKG entity showing the integration of graph structure (a), visual imagery (b), and textual descriptions (c).

Existing methods often struggle with two primary issues:

  1. Over-Homogenization: Many models attempt to "align" modalities by forcing image, text, and graph embeddings into the same mathematical space. This often results in the loss of modality-specific details that are crucial for distinguishing complex relations.
  2. The Few-Shot Bottleneck: Updating massive multimodal models for every new relation is computationally prohibitive and prone to overfitting when data is scarce.

Proposed Methodology: FusionAdapter

FusionAdapter addresses these challenges by building upon the MetaR framework and introducing two key innovations: the Modality Adapter and Contrastive Diversity Loss.

1. The Modality Adapter

Instead of retraining the entire entity embedding, FusionAdapter utilizes a lightweight Adapter Module. This module consists of a feed-forward network with a bottleneck structure inserted into the visual and textual processing streams. During meta-testing—when the model learns a new, unseen relation—only these small adapters are updated. This approach is highly parameter-efficient, adding only 0.22% to 0.75% more parameters to the base model.

The architecture of the Modality Adapter, featuring a bottleneck feed-forward structure to efficiently process modality-specific data.

2. Contrastive Diversity Loss

Perhaps the most significant conceptual shift in this work is the move from alignment to diversity. The authors introduce a diversity loss that penalizes the model if different modalities become too similar. By maintaining a "margin" of difference, the model is forced to extract complementary information—such as relational context from text and visual attributes from images—rather than redundant signals.

The final entity representation is calculated as a summation of the structural, adapted textual, and adapted visual embeddings: $$E = E_{Structure} + E'_{Text} + E'_{Visual}$$

Experimental Evaluation

The researchers evaluated FusionAdapter on three benchmark datasets: WN9-IMG (WordNet-based), FB-IMG (Freebase-based), and FB-IMG (F) (a filtered version to prevent relation leakage).

Key Results

  • Performance Gains: FusionAdapter consistently outperformed state-of-the-art (SOTA) models. On the WN9-IMG dataset, it achieved a Mean Reciprocal Rank (MRR) of 0.327, a significant leap over MetaR’s 0.209.
  • Modality Importance: Ablation studies revealed that text generally provides a larger performance boost than images, likely due to the richer relational semantics found in textual descriptions.
  • Robustness: The model proved resilient to missing data. When an entity lacked an image or text, the adapters helped the model "fill in the blanks" more effectively than unimodal alternatives.

Conclusion

FusionAdapter demonstrates that rigid alignment between modalities can be counterproductive. By allowing different data streams to remain distinct, the model captures a more nuanced understanding of entities and their relations. For the future of knowledge graph completion, this research suggests that the most effective way to fuse data is to ensure that each modality retains its unique "flavor."

Summary updated

Abstract

Multimodal Knowledge Graphs (MMKGs) incorporate various modalities, including text and images, to enhance entity and relation representations. Notably, different modalities for the same entity often present complementary and diverse information. However, existing MMKG methods primarily align modalities into a shared space, which tends to overlook the distinct contributions of specific modalities, limiting their performance particularly in low-resource settings. To address this challenge, we propose FusionAdapter for the learning of few-shot relationships (FSRL) in MMKG. FusionAdapter introduces (1) an adapter module that enables efficient adaptation of each modality to unseen relations and (2) a fusion strategy that integrates multimodal entity representations while preserving diverse modality-specific characteristics. By effectively adapting and fusing information from diverse modalities, FusionAdapter improves generalization to novel relations with minimal supervision. Extensive experiments on two benchmark MMKG datasets demonstrate that FusionAdapter achieves superior performance over state-of-the-art methods.
BibTeX
@misc{liu2025fusionadapter,
  author = {Ran Liu and Yuan Fang and Xiaoli Li},
  title = {{FusionAdapter for Few-Shot Relation Learning in Multimodal Knowledge Graphs}},
  year = {2025},
  eprint = {2510.00894},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.00894},
}
Figure