DKE-Research at SemEval-2025 Task 7: A Unified Multilingual Framework for Cross-Lingual and Monolingual Retrieval with Efficient Language-specific Adaptation

TL;DR

This paper introduces a unified framework for multilingual and cross-lingual fact-checked claim retrieval, tackling the challenge of misinformation spread across languages. By combining contrastive learning with a novel conflict-aware batch sampler and efficient language-specific adapters, the system significantly improves retrieval accuracy, even for unseen languages.

What Problem Does This Paper Address?

The rapid spread of misinformation online is a critical global issue, exacerbated by the fact that false information often transcends language barriers. Fact-checkers face a significant challenge in identifying and verifying claims across diverse languages, making multilingual and cross-lingual Natural Language Processing (NLP) techniques essential. Existing approaches often struggle with the nuances of different languages, leading to potential errors and inefficiencies. This paper aims to bridge this gap by developing a unified system that can effectively retrieve relevant fact-checked claims in both monolingual and cross-lingual settings, addressing the need for scalable and accurate misinformation detection.

Methods

The core of the proposed framework is built upon a Multilingual E5 (M-E5) model, a powerful pre-trained model capable of generating language-agnostic representations. To enhance its performance for fact-checking, the authors introduce several key innovations:

• Contrastive Learning with In-batch Multiple Negative Ranking Loss: This technique trains the model to maximize the similarity between relevant query-document pairs while treating all other documents within the same batch as negative examples. This helps the model learn discriminative representations that capture cross-lingual semantic correspondences.
• Conflict-aware Batch Sampler: A novel component designed to prevent training conflicts. It ensures that each query and document appears only once within a batch, avoiding situations where an instance might inadvertently act as both a positive and a negative example. This guarantees that the "negatives" used in the loss function are truly negative.
• Language-specific Adapters (LADA): To efficiently fine-tune the model for individual languages without modifying the entire base model, the authors employ low-rank adapters. These adapters are trained to inject language-specific knowledge. For unseen languages, they propose a strategy of merging adapters from morphologically similar seen languages.

Datasets

The experiments were conducted using the SemEval 2025 Task-7 dataset. This dataset is a modified version of the MultiClaim dataset and includes:

• A database of fact-checked claims.
• Posts extracted from social media platforms.
• Mappings between posts and claims.
• English translations for non-English content.

The dataset supports both cross-lingual and monolingual fact-checked claim retrieval tasks. The paper provides statistics on the number of queries for each language in both settings.

Key Insights and Ideas

• The importance of truly negative examples: The conflict-aware batch sampler's effectiveness highlights that carefully curated negative examples are crucial for robust contrastive learning, especially in complex multilingual settings.
• Efficient adaptation for unseen languages: The LADA approach, particularly the strategy of merging adapters from similar languages, demonstrates a promising way to extend the model's capabilities to languages with limited training data.
• Unified framework benefits: The paper shows that a single framework can effectively handle both monolingual and cross-lingual retrieval, simplifying the development and deployment of misinformation detection systems.
• M-E5's strong foundation: The M-E5 model, even without the proposed constraints, provides a solid baseline, underscoring the power of large multilingual pre-trained models.

Key Results

The proposed framework, LADA-M-E5, achieved significant improvements across various languages:

• Development Set (Table 2): LADA-M-E5 consistently outperformed the baseline M-E5 in both monolingual and cross-lingual settings. For instance, in the cross-lingual average, LADA-M-E5 achieved 86.1, compared to M-E5's 83.5. The conflict-aware batch sampler also showed improvements in several monolingual languages.
• Test Set (Table 3): For unseen languages (Polish and Turkish), LADA-M-E5 demonstrated substantial gains in monolingual retrieval, with an average of 71.3 compared to M-E5's 70.6. However, a notable discrepancy was observed in the cross-lingual performance for unseen languages between the development and test sets, suggesting potential challenges in cross-lingual alignment for these languages.

Limitations and Future Work

• Cross-lingual alignment for unseen languages: The paper notes a performance discrepancy in cross-lingual retrieval for unseen languages between the development and test sets. This suggests that while language-specific adapters help with monolingual tasks, cross-lingual alignment for these languages might still be a challenge.
• Low-resource languages: The authors express a desire to extend their approach to more low-resource languages.
• Real-world applications: Future work will focus on optimizing retrieval efficiency for real-world applications.

@inproceedings{wang2025dke,
  author = {Yuqi Wang and Kangshi Wang},
  title = {{DKE-Research at SemEval-2025 Task 7: A Unified Multilingual Framework for Cross-Lingual and Monolingual Retrieval with Efficient Language-specific Adaptation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2025},
}