Cross-Lingual Semantic Alignment and Textual Relatedness

31 concepts27 papers1 proposal

Natural language processing has long suffered from an English-centric bias. When a model learns to extract meaning, it often implicitly relies on the specific syntactic and morphological structures of the language it was trained on. However, true semantic understanding should be language-independent: the core meaning of a concept or the logical relatedness between two ideas remains constant whether expressed in English, Amharic, or Indonesian.

The fundamental challenge of cross-lingual semantic understanding is to decouple meaning from language-specific syntax, allowing models to align and compare concepts across the global linguistic landscape.

De-Biasing Meaning from Syntax

Traditional semantic parsers struggle when applied to typologically diverse languages because they are optimized for English word order and structure. To build intuition, think of a semantic representation as a language-independent blueprint of a sentence's core message. As highlighted in the Proceedings of the First International Workshop on Designing Meaning Representations, the goal is to reconcile language-specific nuances with universal semantic principles.

Daniel Hershcovich et al. demonstrated this in SemEval-2019 Task 1: Cross-lingual Semantic Parsing with UCCA. By mapping sentences into the Universal Conceptual Cognitive Annotation (UCCA) framework, the authors showed that a structurally abstract, coarse-grained semantic framework allows models to transfer parsing capabilities from data-rich languages to low-resource ones without getting tripped up by structural variations. Similarly, early foundational work by Hammam R. Yusuf in An Analysis of Indonesian Language for Interlingual Machine-Translation System emphasized that robust multilingual translation requires analyzing source-specific morphology to produce a genuinely language-independent interlingual representation.

Grounding Words in Context Across Languages

How do we evaluate whether a model truly understands a word's meaning across languages? The Multilingual and Cross-lingual Word-in-Context (MCL-WiC) task requires a model to determine if a target word shares the same sense across two different sentences, which may be written in different languages.

To solve this, models must move beyond static word embeddings. Huiling You et al. evaluated this in Uppsala NLP at SemEval-2021 Task 2: Multilingual Language Models for Fine-tuning and Feature Extraction in Word-in-Context Disambiguation, finding that directly fine-tuning massive multilingual models like XLM-RoBERTa significantly outperforms using them as static feature extractors, particularly in cross-lingual scenarios.

But how do we inject explicit sense knowledge into these embeddings? Maxim Rachinskiy and Nikolay Arefyev introduced a powerful intuition in GlossReader at SemEval-2021 Task 2: Reading Definitions Improves Contextualized Word Embeddings: they proposed Gloss Language Modeling (GLM). By training a model to explicitly align dictionary definitions (glosses) with contextualized representations, the model learns a language-agnostic semantic space. This allows it to excel at cross-lingual disambiguation without needing language-specific resources beyond English definitions.

To optimize this further, Adis Davletov et al. in LIORI at SemEval-2021 Task 2: Span Prediction and Binary Classification approaches to Word-in-Context Disambiguation framed the problem as a binary classification task using pooled XLM-R representations, proving that large-scale cross-lingual models can effectively compute semantic overlap when properly aggregated.

Quantifying Sentence-Level Relatedness Globally

Moving from individual words to full sentences increases the complexity. Semantic Textual Relatedness (STR) requires models to quantify the degree of meaning shared between two sentences, moving far beyond simple word-overlap metrics. As noted by Srushti Sonavane et al. in CAILMD-23 at SemEval-2024 Task 1: Multilingual Evaluation of Semantic Textual Relatedness, while STR is well-developed for English, evaluating it across diverse, low-resource languages is essential for true multilingual NLP.

The SemEval-2024 Task 1 benchmark pushed this boundary across 14 languages. Researchers tackled this using several distinct paradigms:

  1. Architectural Fusion: Jesus German Ortiz Barajas et al. introduced Sentence-CROBI in MBZUAI-UNAM at SemEval-2024 Task 1: Sentence-CROBI, a Simple Cross-Bi-Encoder-Based Neural Network Architecture for Semantic Textual Relatedness. Bi-encoders are fast but miss token-level interactions between sentences; cross-encoders capture these interactions but are computationally heavy. By combining them into a cross-bi-encoder architecture with multilingual fine-tuning, the authors built a model highly adaptable to low-resource languages.
  2. Translation vs. Direct Fine-Tuning: Shubhashis Roy Dipta and Sai Vallurupalli explored this trade-off in UMBCLU at SemEval-2024 Task 1: Semantic Textual Relatedness with and without machine translation, demonstrating that while machine translation can bridge the gap to high-resource models, direct fine-tuning on native low-resource data captures unique cultural and idiomatic nuances that translation often obliterates.
  3. Ensemble and Domain Adaptation: Systems like the one proposed by Nguyen Tuan Kiet and Dang Van Thin in NRK at SemEval-2024 Task 1: Semantic Textual Relatedness through Domain Adaptation and Ensemble Learning on BERT-based models leveraged domain-adapted BERT variants combined with weighted voting to handle complex cross-cultural contexts, such as Algerian Arabic and Amharic.

This sentence-level alignment also extends to logical plausibility. Hao Yang et al. demonstrated in HW-TSC at SemEval-2022 Task 3: A Unified Approach Fine-tuned on Multilingual Pretrained Model for PreTENS that sentence acceptability and taxonomic contradictions can be modeled uniformly across languages (English, French, Italian). By converting regression scores into probabilistic classification labels, they ensured that a model's grasp of logical plausibility remains invariant to syntax.

Bridging Lexical Gaps via Reverse Dictionaries and Concept Vectors

A final frontier in this space is mapping continuous semantic vectors back to human-readable concepts or words across languages.

Consider the "Reverse Dictionary" task: given a definition in one language, can a model predict the corresponding word vector in another? Bin Li et al. solved this in LingJing at SemEval-2022 Task 1: Multi-task Self-supervised Pre-training for Multilingual Reverse Dictionary by pairing a multilingual DeBERTa backbone with a two-stage self-supervised regimen consisting of Masked Language Modeling and Contrastive Learning. This forced the model to map descriptive human definitions directly to precise, language-agnostic lexical spaces.

Taking this a step further, Simone Conia and Roberto Navigli developed "Conception" in Conception: Multilingually-Enhanced, Human-Readable Concept Vector Representations. Instead of aligning words, the authors explicitly modeled concepts using multilingual knowledge graphs. This approach creates vector representations that are inherently language-independent and human-readable, significantly boosting performance in semantic similarity and word sense disambiguation for low-resource languages.

Key Foundational Work

Proceedings of the First International Workshop on Designing Meaning Representations (2019) — contributed 3 concept elements to this direction.
SemEval-2019 Task 1: Cross-lingual Semantic Parsing with UCCA (2019) — contributed 1 concept elements to this direction.
An Analysis of Indonesian Language for Interlingual Machine-Translation System (1992) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_cross_lingual_semantic_understanding_e327d512,
  title = {cross-lingual semantic understanding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-lingual-semantic-understanding/rdr_e327d512d7ae48b3a8742612c009ac5e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_cross_lingual_semantic_understanding_e327d512_v1,
  title = {cross-lingual semantic understanding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-lingual-semantic-understanding/rdr_e327d512d7ae48b3a8742612c009ac5e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake