DLIR: Spherical Adaptation for Cross-Lingual Knowledge Transfer of Sociological Concepts Alignment

Zeqiang Wang, Jon Johnson, Suparna DeEMNLP 2025

TL;DR

This paper introduces DLIR, a novel method for aligning complex sociological concepts across languages, even with limited parallel data. By disentangling domain-specific meaning from language-specific nuances using a dual-branch LoRA architecture and spherical geometry, DLIR achieves state-of-the-art cross-lingual concept retrieval and demonstrates strong knowledge transfer capabilities.

What Problem Does This Paper Address?

Cross-cultural research, especially in fields like sociology, often requires aligning concepts that are deeply embedded in specific cultural and linguistic contexts. Terms like "social inequality" or "unemployment" can carry subtly different meanings across languages and societies, making direct translation or simple keyword matching insufficient. Existing methods struggle with this nuance, particularly for low-resource languages where large parallel datasets are scarce. This lack of robust cross-lingual concept alignment hinders comparative studies, longitudinal research, and the harmonization of data across different linguistic communities.

Methods

DLIR tackles this challenge by proposing a "Vector Decomposition Hypothesis," suggesting that text embeddings can be separated into a domain-specific semantic core and a language-specific component. To achieve this, DLIR employs a Dual-Branch LoRA (Low-Rank Adaptation) architecture:

  • Task Branch: This branch focuses on capturing the core sociological semantics, primarily trained on English data structured by the ELSST hierarchy.
  • Invariance Branch: This branch learns to counteract language-specific variations. It uses Spherical Noise Injection, modeling language perturbations as shifts on a unit hypersphere. These perturbations are learned using Gaussian Mixture Models (GMMs) fitted on minimal parallel concept data, capturing systematic, geometric language differences.

The overall training objective combines a Hierarchical Ranking Loss (to preserve semantic hierarchy) and a Spherical Invariance Loss (to enforce language-agnostic representations).

Datasets

  • ELSST (European Language Social Science Thesaurus): Used for structuring the semantic space and providing concept hierarchies. Minimal parallel data from ELSST was used for offline GMM fitting.
  • Synthetic Descriptive Texts: Generated using DeepSeek-V3, these texts describe sociological concepts without explicitly naming them, encouraging implicit referencing. This dataset was used for core training and evaluation. Human evaluation by domain experts confirmed its quality.
  • Translated ELSST Data: Used for training baseline models (LoRA-FR/DE) for comparison.

Key Insights and Ideas

  • Disentangling Semantics and Language: The core idea that domain knowledge and language-specific features can be separated in embeddings is crucial. DLIR provides empirical support for this via GMM analysis of spherical differences.
  • Spherical Geometry for Perturbations: Modeling language variations as geometric shifts on a hypersphere, rather than simple Euclidean subtraction, allows for a more accurate capture of systematic linguistic differences.
  • GMMs for Language Perturbations: Using GMMs to model these spherical shifts provides a generative model for sampling realistic language perturbations, leading to more effective invariance learning.
  • Parameter Efficiency: DLIR achieves state-of-the-art results by adapting a smaller base model (E5-small) with minimal LoRA updates, demonstrating high efficiency compared to larger models.

Key Results

  • State-of-the-Art Cross-Lingual Retrieval: DLIR achieved the best performance on cross-lingual concept retrieval across 10 languages, outperforming strong zero-shot baselines like E51, KaLM, and Ada-002. For example, it achieved an R@1 score of 0.821, compared to E51's 0.794.
  • Effective Knowledge Transfer: DLIR, trained only on English data, significantly outperformed monolingual LoRA models fine-tuned on translated French and German data. For instance, it achieved a 27.9% higher R@1 score on French compared to LoRA-FR.
  • Ablation Studies: Each component of DLIR contributed to performance gains, with the GMM-based spherical invariance providing the largest boost over generic Gaussian noise.

Limitations and Future Work

  • GMM Fitting Dependency: The effectiveness of DLIR relies on the quality of GMMs fitted from minimal parallel data. Performance might vary for languages with extremely sparse data, and more robust density estimation techniques could be explored.
  • Synthetic Data Reliance: The core training and evaluation heavily depend on synthetic data. While validated, it may not fully capture the nuances of authentic sociological discourse.
  • Limited Language Scope: The current evaluation covers 10 European languages and doesn't include typologically distant languages.
  • Future Work: The authors suggest extending the framework to other specialized domains, enhancing robustness, and exploring direct transfer between non-English pairs. They also propose a comprehensive qualitative error analysis across languages.

Summary updated

BibTeX
@inproceedings{wang2025dlir,
  author = {Zeqiang Wang and Jon Johnson and Suparna De},
  title = {{DLIR: Spherical Adaptation for Cross-Lingual Knowledge Transfer of Sociological Concepts Alignment}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure