Many cross-lingual tools still need a human-made seed dictionary: a few thousand verified word translations used to rotate one language's word vectors onto another's. Fully unsupervised replacements exist, but on distant language pairs they tend to settle on a wrong answer and stay there. The published failures are stark. An adversarial aligner reaches 0.0% top-1 translation accuracy on English–Finnish and English–Estonian (On the Limitations of Unsupervised Bilingual Dictionary Induction). The usual rescue, seeding with identically spelled words such as "Paris" or "taxi," cannot work when the two languages use different writing systems.
We propose to treat this failure as an optimization problem, and to change what the optimizer sees rather than how it searches. The method has two steps. First, blur each language's word-similarity structure by diffusing it over its own nearest-neighbour graph, so that only coarse topical communities remain. Match the two languages at that coarse level. Second, reduce the blur step by step, warm-starting each finer match from the coarser one, until the matching uses the raw similarities. The bet is that the coarse structure of two languages is much more alike than their fine structure. If so, the coarse matching problem has one broad basin, and following it down to full resolution avoids the traps that capture current methods.
Core Claim
Relational alignment asks which word in language B plays the same role as a given word in language A, judged only by how each word relates to the other words in its own language. Gromov–Wasserstein optimal transport, a way of matching two point sets using only their internal distances, is the cleanest version of this (Gromov-Wasserstein Alignment of Word Embedding Spaces). That paper matches the 20,000 most frequent words of each language by comparing intra-language cosine-distance matrices, then fits an orthogonal rotation to the resulting soft matching. It avoids the oscillation of adversarial training. It remains, however, a non-convex quadratic assignment problem, and its iterations are local steps that can stop at a poor matching.
Our claim is specific and falsifiable. For typologically distant pairs, a measure of how differently two languages' words relate to one another (non-isomorphism) is concentrated at fine scales and shrinks as the similarity graphs are diffused. A coarse-to-fine continuation over diffusion time will therefore reach the best-scoring basin of the full objective far more reliably than a single solve, random restarts, or annealing of the entropy parameter. It will also produce a correct dictionary on cross-script pairs without any spelling overlap.
The motivating evidence comes from On the Limitations of Unsupervised Bilingual Dictionary Induction. That paper found that similarity of graph-Laplacian eigenvectors, computed from nearest-neighbour graphs, correlates at $\rho = 0.89$ with unsupervised induction accuracy. Our method turns this diagnostic into a control knob. Diffusion damps the high-frequency Laplacian modes first and keeps the low-frequency ones.
Relation to Prior Work
Three lines of work attack entrapment in seed-free lexicon induction, and all three keep the landscape fixed. A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings builds an initial dictionary from sorted similarity rows. It then escapes bad fixed points by randomly dropping entries of the similarity matrix and by restricting early rounds to frequent words. The Gromov–Wasserstein approach replaces the adversarial game with transport. DM-BLI: Dynamic Multiple Subspaces Alignment for Unsupervised Bilingual Lexicon Induction splits the vocabulary into semantic clusters, aligns them with Wasserstein contrastive learning, and lets words migrate between clusters. It reports a 6.26-point Precision@1 gain on Japanese–English over the previous best. Each of these uses one fixed scale of relational structure. Our proposal instead builds a family of objectives indexed by scale.
DM-BLI is the closest in spirit, because it also decides that local geometry disagrees across languages. It handles this with hard, heuristic clusters and a multi-stage loop. We use one continuous objective whose scale parameter plays the role of cluster granularity.
The optimization literature supplies the method and also limits what we may claim. A Theoretical Analysis of Optimization by Gaussian Continuation smooths an objective by convolving it with a Gaussian in parameter space and bounds the quality of the final solution. Our smoothing acts on the data, the relational matrices, not on the objective's parameters. That bound therefore does not transfer, and we claim no guarantee.
The same paper, together with Continuation Path Learning for Homotopy Optimization, warns about two practical failure modes. The first is schedule sensitivity. The second is that the continuation path may split or jump as the smoothing is reduced. We measure both directly rather than assume a smooth path.
Continuation Path Learning trains a network to output the solution at every smoothing level. It reports a final loss of 0.006 versus 12.63 for gradient descent on the Ackley function. We treat it as a possible extension, not the core, because a stepwise warm-started schedule is the simpler thing to test first.
Research Program
For each language we take the 20,000 most frequent fastText vectors and build a symmetric $k$-nearest-neighbour cosine graph with normalized Laplacian $L$. We compute its bottom $m$ eigenpairs $(U, \Lambda)$, with $m$ in the low hundreds. The diffused similarity at time $t \ge 0$ is $K_t = U e^{-t\Lambda} U^\top$, and $D_t$ is the matching diffusion distance matrix. At each $t$ we solve the entropic Gromov–Wasserstein problem
$$\Gamma_t = \arg\min_{\Gamma \in \Pi(p,q)} \sum_{i,j,k,l} \big(D^{(s)}_{t,ik} - D^{(t)}_{t,jl}\big)^2 \Gamma_{ij}\Gamma_{kl} - \varepsilon H(\Gamma).$$
Here $\Gamma$ is the soft matching between source and target words, and $\Pi(p,q)$ is the set of matchings with frequency-based marginals $p, q$. The superscripts $(s)$ and $(t)$ denote source and target. $H$ is entropy and $\varepsilon$ is fixed.
We start at a large $t$, where only a few communities survive. We step $t$ down geometrically, initializing each solve with the previous $\Gamma$, and finish with the raw cosine distances used in the original Gromov–Wasserstein alignment paper. Because $K_t$ has rank $m$, the gradient of the quadratic term can be computed from low-rank factors, so each solve stays tractable on one GPU. The final matching becomes pseudo-supervision for orthogonal Procrustes over the full vocabulary. Retrieval uses Cross-domain Similarity Local Scaling (CSLS), a nearest-neighbour score that corrects for "hub" words that are close to everything. This follows the standard pipeline in artetxem/vecmap.
Alongside the method we run an analysis that can falsify the premise before any accuracy numbers matter. For each pair and each $t$ we compute two quantities. The first is eigenvector similarity between the two diffused graphs. The second is the Gromov–Wasserstein cost of the gold matching implied by the MUSE test dictionary, compared with the cost of the best matching our solver finds. The first shows whether diffusion makes the languages look more alike. The second shows whether the true answer is actually a low point of the smoothed objective.
We also record the matching change $\|\Gamma_t - \Gamma_{t'}\|$ between consecutive scales, to locate any splitting or jumping of the path.
As a downstream test of whether the seed-free dictionary can stand in for human supervision, we plug the resulting mapping into Embedding-Enhanced GIZA++: Improving Low-Resource Word Alignment Using Embeddings. That system originally used a VecMap mapping as its source of translation similarities. We measure whether better lexicon induction reduces token-level alignment error on tiny bitexts.
Baselines
The naive reference point is plain entropic Gromov–Wasserstein at $t = 0$ from a uniform initialization, exactly as in Gromov-Wasserstein Alignment of Word Embedding Spaces, with the same vocabulary, $\varepsilon$, Procrustes refinement and CSLS retrieval. That paper reports roughly 81–82% Precision@1 on MUSE English–Spanish and English–French. It describes robust English–Finnish alignment on the Dinu benchmark, but the file gives no figure for that pair, so we rerun it. This comparison isolates the effect of continuation alone.
The strongest seed-free competitor on distant pairs is VecMap's robust self-learning (A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings). It reports 33.50% on Dinu English–Finnish and 37.33% on English–Spanish, where earlier unsupervised methods scored below 2% on English–Finnish. It is run from the public code with default settings on identical embeddings.
DM-BLI is the strongest published method on cross-script MUSE pairs. It reports a 6.26-point Precision@1 gain over CL-BLI on Japanese–English. No absolute Japanese–English Precision@1 was found on Lacuna for DM-BLI, so we rerun it where code allows and report its numbers ourselves.
The weakly supervised identical-word seed from On the Limitations of Unsupervised Bilingual Dictionary Induction reports 28.0% Precision@1 on English–Finnish. It serves as a reference on shared-script pairs only.
Two ablations isolate the key ingredient. The first anneals the entropy weight $\varepsilon$ from large to small at $t = 0$. The second applies parameter-space smoothing, perturbing the matching with Gaussian noise whose variance shrinks, in the style of Gaussian continuation. Both are also coarse-to-fine schedules, so beating them is what shows that data-side diffusion, rather than any schedule, does the work. A third ablation replaces graph diffusion with truncation to the top-$m$ principal components of the embeddings, to test whether the graph matters at all.
Every method gets the same embeddings, the same vocabulary, and the same number of Gromov–Wasserstein solves or wall-clock budget. Random restarts are matched to the number of continuation steps.
Beating the baselines means two things. On distant pairs (English–Finnish, English–Estonian, English–Japanese, English–Chinese, English–Russian) the method must exceed the best seed-free baseline in median Precision@1 across ten seeds. It must also have a lower failure rate, defined as the fraction of seeds below 5%. On close pairs it must stay within one point of plain Gromov–Wasserstein.
Why not just…?
Why not just use random restarts and keep the run with the lowest objective? This is the most serious threat. If the true matching already lies in the deepest basin of the raw objective, enough restarts will find it, and continuation only saves compute. Our basin experiment settles this. We run 100 restarts per pair and plot final objective against Precision@1. If the lowest-objective restart is reliably correct, continuation's value shrinks to efficiency and we will report that. If low objective and accuracy come apart at $t = 0$ but agree at coarse $t$, restarts cannot fix the problem and continuation can.
Why not just anneal the entropy parameter, which is already a homotopy? Annealing $\varepsilon$ smooths the matching toward uniform, but it leaves the relational matrices untouched. The fine-scale disagreement between the languages therefore stays in the objective at every stage. It is a real alternative, so it is a matched ablation. If it equals diffusion continuation, the claim that non-isomorphism is concentrated at fine scales is not what drives the gains.
Why not just use VecMap or DM-BLI, which already escape local traps? Their published gains are real, but their escape mechanisms are fixed-scale heuristics: random dropout, frequency cutoffs, and hard clusters. The available numbers leave distant pairs far from solved, for example 33.50% on English–Finnish. We compare against both directly. We also check whether our dictionary is a better initialization for VecMap's self-learning, which would make the methods complementary rather than competing.
Why not just seed with identical strings or a large language model? Identical strings do not exist across scripts, as On the Limitations of Unsupervised Bilingual Dictionary Induction itself implies. DM-BLI reports being more robust than zero-shot GPT-3.5 and Llama prompting in its setting. The cross-script pairs are exactly where our method must prove its worth.
Proposed Experiments
The first check comes before any accuracy comparison. On MUSE English–Finnish, English–Estonian, English–Japanese and English–Chinese, we plot eigenvector similarity and the gold-matching cost gap against diffusion time. The premise holds only if both improve monotonically as $t$ grows while the gold matching stays near-optimal at coarse scales. English–Spanish serves as a control where little change is expected. If distant pairs are no more alike at coarse scales, we stop and report a negative result on the scale hypothesis.
The second check is the basin study. For each pair we run 100 random initializations of plain Gromov–Wasserstein, show the spread of objectives and Precision@1, and report what fraction of continuation runs land in the best basin.
The third is the main comparison on MUSE and Dinu. It covers the close pairs (English–Spanish, English–French, English–Italian), the morphologically rich pairs (English–Finnish, English–Estonian), and the cross-script pairs (English–Japanese, English–Chinese, English–Russian). We report Precision@1 and Precision@5 as medians and failure rates over ten seeds. We also report the path-jump statistic, to show where along the schedule the matching reorganizes and whether jumps predict failures.
The fourth check tests whether the seed-free dictionary removes the need for supervision downstream. We substitute our mapping for VecMap's inside Embedding-Enhanced GIZA++ in its lowest-resource setting, aligning only the few-hundred-line test bitexts for Romanian–English, German–English and English–French. The reference is its reported alignment-error reductions of 8.5, 10.9 and 12.0 points over plain GIZA++. The comparison is judged on alignment error rate against the gold alignments. Success means matching those gains with a mapping that never saw a seed. We would like to also test a cross-script bitext, but the file lists no gold-aligned cross-script bitext, so this remains conditional on finding one.
References
- On the Limitations of Unsupervised Bilingual Dictionary Induction
- Gromov-Wasserstein Alignment of Word Embedding Spaces
- A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings
- DM-BLI: Dynamic Multiple Subspaces Alignment for Unsupervised Bilingual Lexicon Induction
- A Theoretical Analysis of Optimization by Gaussian Continuation
- Continuation Path Learning for Homotopy Optimization
- Embedding-Enhanced GIZA++: Improving Low-Resource Word Alignment Using Embeddings