Where a Word Sits in Two Corpora: Frequency-Placed Hyperbolic Bridges for Cross-Language Citation

Papers written in Chinese, Russian or Japanese are cited by English papers for the same reasons English papers are cited: for methods, for background, and for comparing results. Yet citation indexes lose track of most of them. Among the cross-lingual citations in 1.1 million English arXiv papers, only 11.2% resolve to a record in the global academic graph, against 42.6% of references overall (Cross-Lingual Citations in English Papers). One partial fix is citation recommendation: given a Chinese paper, suggest the English papers it should cite, or the reverse. The strongest system of this kind builds one graph containing papers, authors, venues and machine-translated keywords from both languages, then learns flat vector representations from random walks over it. Every translated keyword counts as an equally good bridge between the two languages.

We propose to stop treating bridge words as equal, and to use a word's standing in each corpus to decide where it goes in a curved representation space. The rule is simple. Count how often a bridge word occurs in the regional corpus and how often its translation occurs in the global one. Words common in both are general concepts. Words common locally but rare globally are the regional specialties that translation tends to mangle, and they are also the reason anyone reads regional work. Words rare in both are mostly noise. We embed the graph in hyperbolic space, a geometry whose room grows exponentially with distance from the center, so it fits tree-like, power-law data in few dimensions. General words are pushed toward the center. Regional specialties are placed far out, where they get room of their own instead of being crowded into their nearest English neighbor. Words rare in both corpora are removed.

Core Claim

In cross-language citation graphs, the gap between a keyword's regional and global frequency tells you how general or specific that keyword is. That signal should set the keyword's distance from the center of a hyperbolic embedding. It should not be used as a cutoff for pruning. We predict that this combination beats both the Euclidean state of the art and a plain hyperbolic swap. We also predict that most of the gain will come from regional papers whose keywords are frequent locally and rare globally. A Euclidean model given the same frequency weights should not recover that gain, because the benefit comes from the extra room hyperbolic space has far from its center.

Positioning Against Prior Work

The work that motivates and bounds this proposal comes in three groups.

The first group supplies the task and the baseline to beat. Cross-language Citation Recommendation via Hierarchical Representation Learning on Heterogeneous Graph (HRLHG) and its companion Cross-language Citation Recommendation via Publication Content and Citation Representation Fusion build a Wanfang–ACM graph with Google-translated keyword bridges. HRLHG learns which types of edges are useful and embeds its random walks with skip-gram, a word2vec-style objective, in Euclidean space. Its "hierarchy" refers to two levels of walk sampling, not to the geometry of the space, and it weights all bridges uniformly. We keep its graph and evaluation protocol and change both the geometry and how bridges are treated.

The second group supplies the method. Hyperbolic Heterogeneous Information Network Embedding (HHNE) shows that bibliographic networks with several node types embed well in the Poincaré ball, one standard model of hyperbolic space. It reports about 0.87 link-prediction AUC in two dimensions on DBLP, against about 0.67 for Euclidean models. HHNE is monolingual and depends on hand-specified walk patterns. Poincaré Embeddings establishes that distance from the center encodes generality. Cross-lingual Word Embeddings in Hyperbolic Space shows the same holds for bilingual word embeddings (HyperLex Spearman 0.179 cross-lingual), but only for words, not documents or citations. We use the Lorentz model, a second representation of hyperbolic space, because Fast and Geometrically Grounded Lorentz Neural Networks argues it avoids the precision collapse the Poincaré ball suffers near its boundary. That matters here because regional specialties will sit far from the center.

The third group constrains the design. The frequency-filtering literature prunes the low-frequency tail as noise, as the Estonian paradigm lexicon does (Eesthetic) and as veterinary record mining does (Text mining for disease surveillance in veterinary clinical data). Applied to one pooled corpus, that approach would remove exactly the regional terms we care about, the "long-tail erasure" problem. Our two-corpus comparison is designed to avoid this. Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic Spaces warns that separately trained hyperbolic spaces can come out with their branches shuffled, and that Euclidean alignment fails on them (1.4% vs 88.8% P@10 on English–Spanish WordNet). We sidestep that problem by training a single shared space over the joint graph rather than aligning two spaces.

Research Program

For each translated bridge keyword $w$, let $f_R(w)$ be its document frequency in the regional corpus (Wanfang) and $f_G(w)$ the document frequency of its translation in the global corpus (ACM), both normalized by corpus size. We compute two quantities. The first is a specificity score:

$$s(w) = \log\frac{f_R(w)+\alpha}{f_G(w)+\alpha},$$

where $\alpha$ is a small smoothing constant. The second is joint support, $m(w)=\min(f_R(w), f_G(w))$ combined with $f_R(w)$. Keywords whose regional frequency falls below a threshold $\tau$ are removed; these are the words rare in both corpora or rare locally. For each remaining keyword we set a target distance from the center, $\rho(w)$, that increases monotonically with $s(w)$. We then add a soft penalty $\lambda\,(d_{\mathbb{L}}(o, x_w) - \rho(w))^2$ to a hyperbolic skip-gram objective trained on HRLHG-style walks. Here $x_w$ is the keyword's embedding, $o$ is the center of the Lorentz model, $d_{\mathbb{L}}$ is hyperbolic distance, and $\lambda$ sets the penalty's strength. Paper, author and venue nodes are left free. We expect heavily cited papers to drift toward the center, as HHNE observed.

Bridge edges also get walk-transition weights that decrease with $s(w)$ at intermediate values and rise again for strongly supported regional specialties. This way random walks cross between languages through general bridges often, and through specific ones when those specific bridges are well supported. Retrieval ranks candidates by negative hyperbolic distance.

One analysis is required alongside the model. A high $s(w)$ can mean a genuinely regional concept or a bad translation. We will hand-annotate 300 high-score keywords, sampled across score bins, as regional concept, mistranslation, or other. We will then report whether our weighting treats the two classes differently. If mistranslations dominate the high-score bin, the claim needs a translation-confidence term, and we will say so.

Baselines

The naive reference points are text-only retrieval using BM25 over Google-translated titles and abstracts, and HRLHG's own node2vec-on-full-graph baseline. HRLHG reports the latter at MAP@10 0.3661 on Wanfang–ACM.

The strongest existing method is HRLHG itself, at a reported MAP@10 of 0.4033 on the same dataset (Cross-language Citation Recommendation via Hierarchical Representation Learning on Heterogeneous Graph). We will reimplement it on the same graph and splits. We will report both our reproduced number and the published one, and compare against whichever is higher.

A modern multilingual dense retriever is the second strongest competitor. This is a bi-encoder that embeds the original Chinese and English text directly into one space, with no translation. HRLHG only compared against 2018-era translation pipelines, so this retriever may be much stronger than anything HRLHG faced. No reported result was found on Lacuna for any such model on this dataset. We will run it with the same candidate pool, fine-tuned on the training citations with the same tuning budget as our model.

Three ablations isolate our key ingredient. The first is HHNE-style hyperbolic embedding of the unmodified HRLHG graph, which adds curvature but no frequency information. The second is Euclidean HRLHG with our pruning and frequency-derived edge weights, which adds frequency information but no curvature. The third is our model with the specificity score replaced by pooled single-corpus frequency pruning, the classic filter.

All models get the same graph, the same dimension sweep from 2 to 128, and an equal number of hyperparameter trials and GPU hours. Beating the field means exceeding the best of HRLHG and the dense retriever on MAP@10 and NDCG@10, with significance over five seeds, and beating both single-ingredient ablations.

Why not just…?

Why not just use a multilingual dense retriever on raw text? This is the real threat. The evidence against it in the file is dated: HRLHG beat translation-based baselines, and the SS-MONO taxonomy study (Geometric Constraints for Small Language Models) found large language models underperformed a geometry-constrained small model on hierarchy placement. Neither result tests a fine-tuned multilingual retriever on citation recommendation. We therefore run one directly. We also run a late-fusion hybrid of the retriever with our graph model. If the retriever alone matches the hybrid, the graph contributes nothing and we will report that. If the hybrid wins mainly on regional-specialty queries, the graph is doing the job we claim for it.

Why not just keep HRLHG and weight bridges by IDF? A frequency weight might be all that matters. The Euclidean-with-frequency-weights ablation tests exactly this. Our claim fails if that ablation comes within noise of the full model, especially in low dimensions where hyperbolic room should matter most.

Why not just use HHNE on the existing graph? Curvature alone may capture most of the gain, since HHNE's DBLP results show a large gap at two dimensions. The hyperbolic-without-frequency ablation measures this. The prediction that separates the two is where the gain lands: our model should improve specifically on queries dominated by high-$s(w)$ keywords, while plain HHNE should not.

Why not just raise Euclidean dimension? We sweep dimensions for every model, so the Euclidean ceiling at 128 dimensions is measured rather than assumed.

Proposed Experiments

The primary experiment uses the Wanfang–ACM bilingual dataset: 14,631 Chinese papers, 248,893 English papers and 27,101 Chinese-to-English ground-truth citations, under HRLHG's protocol. We report MAP@10, NDCG@10, Precision@10 and MRR. Access to this dataset depends on its authors releasing it; if they do not, we will rebuild a comparable graph and cannot quote the published numbers.

The decisive check is a stratified analysis. We bucket test queries by the mean specificity score of their keywords and plot each model's gain over HRLHG per bucket. The claim predicts that the gain rises with specificity, and that it appears for the hyperbolic frequency-placed model but not for either single-ingredient ablation.

A geometry check asks two questions. Does the learned distance from the center of free paper nodes correlate with citation count? Do annotated regional concepts end up farther out than annotated mistranslations?

A monolingual sanity check uses the unarXive 2022 citation-recommendation task (unarXive 2022, 2.5 million paragraph–marker pairs). The radial prior degenerates there because both corpora are the same, so the model should do no worse than HHNE. This confirms the curvature and optimization choices do not cost anything when no language barrier exists.

A negative result on the stratified analysis would settle the matter against us. That result is gains that are flat across buckets, or gains matched by Euclidean frequency weighting. It would show that frequency and curvature do not combine in the way argued here.