Abstract
We propose a novel geometric approach for learning bilingual mappings given monolingual embeddings and a bilingual dictionary. Our approach decouples learning the transformation from the source language to the target language into (a) learning rotations for language-specific embeddings to align them to a common space, and (b) learning a similarity metric in the common space to model similarities between the embeddings. We model the bilingual mapping problem as an optimization problem on smooth Riemannian manifolds. We show that our approach outperforms previous approaches on the bilingual lexicon induction and cross-lingual word similarity tasks. We also generalize our framework to represent multiple languages in a common latent space.
Introduction
In Natural Language Processing, word embeddings represent words as vectors in a high-dimensional space, capturing semantic relationships within a single language. However, vector spaces for different languages (e.g., English and French) are not naturally aligned. To enable cross-lingual tasks, researchers typically seek a transformation matrix $W$ to map one space to another. While many methods assume language spaces are "isomorphic" and use simple orthogonal rotations, languages in reality are rarely perfectly identical in shape. This work introduces a flexible alignment method that handles feature correlations and scales to multiple languages simultaneously.
Methodology: The GeoMM Approach
The proposed Geometry-aware Multilingual Mapping (GeoMM) approach factorizes the transformation matrix $W$ as: $$W = U_t B U_s^\top$$
- Language-Specific Rotations ($U_s, U_t$): Orthogonal matrices that rotate source and target embeddings into a common latent space.
- Shared Mahalanobis Metric ($B$): A positive definite matrix that accounts for feature correlations and variances, "warping" the latent space to reflect true semantic similarity.
Unlike traditional regression-based methods, GeoMM treats alignment as a binary classification task, learning to distinguish between correct translation pairs and random pairs. Optimization is performed using Riemannian conjugate gradient descent directly on the manifolds defined by the orthogonality and positive-definiteness constraints, ensuring a more principled and efficient search for the optimal transformation.
Experimental Results
The model was evaluated on standard benchmarks including MUSE and VecMap.
Bilingual Lexicon Induction (BLI)
On the MUSE dataset, GeoMM achieved state-of-the-art performance, significantly outperforming the standard Procrustes baseline and other multi-step frameworks.
As shown in the results above, GeoMM reached an average Precision@1 of 70.0%, compared to 66.9% for the Procrustes method. Similar gains were observed on the VecMap dataset, where GeoMM achieved an average P@1 of 43.3%.
Ablation Study
To understand the contribution of each component, the authors conducted an ablation study on the VecMap dataset.
The results demonstrate that both the rotations and the Mahalanobis metric are vital. Removing the metric (setting $B=I$) or the rotations led to substantial performance degradation, with accuracy dropping by over 30% in some cases (e.g., row 3 in the table).
Conclusion
GeoMM provides a robust geometric framework for multilingual embedding alignment by decoupling global orientation from local similarity measures. The joint learning of a shared latent space not only improves bilingual translation but also enhances "one-hop" indirect translation between languages without direct dictionaries. Future work includes exploring fully unsupervised extensions and integrating advanced retrieval metrics like CSLS directly into the optimization objective.
Summary updated