Cross-Lingual Citations in English Papers: A Large-Scale Analysis of Prevalence, Usage, and Impact

Tarek Saier, Michael Färber, Tornike Tsereteli

Scientific discourse is increasingly global, yet citation data often remains "Anglocentric." This study, published in the International Journal on Digital Libraries, investigates the prevalence and nature of cross-lingual citations—instances where English-language papers cite works written in other languages. By analyzing over 1.1 million documents, the researchers provide insights into how these citations bridge "language silos" in the scientific community.

Defining the Cross-Lingual Bridge

A cross-lingual citation occurs when a citing document (in English) references a cited document written in a different language. These citations are often identified by explicit language markers in the reference section, such as "(in Russian)" or "(in Chinese)".

Terminology and conceptual diagram of cross-lingual citations.

The study addresses three core questions regarding these citations:

  1. Prevalence: How common are they across different scientific fields?
  2. Usage: Do authors cite non-English works for different reasons than English ones?
  3. Impact: How visible are these citations to standard scholarly metrics and discovery tools?

Methodology: Mining the unarXive

The researchers utilized the unarXive dataset, comprising 1.1 million English papers from 1992 to 2019. To identify cross-lingual citations at scale, they developed a high-precision methodological proxy based on explicit language markers. This method was validated using automated language detection (Lingua) and manual labeling. Furthermore, they employed SciBERT to classify citation intent (e.g., Background, Method) and sentiment, allowing for a direct comparison between cross-lingual and monolingual citation behaviors.

Key Findings

The study identified approximately 33,290 cross-lingual references across 18,171 documents. Roughly 1 in every 66 English papers contains at least one explicit cross-lingual citation. However, prevalence varies significantly by discipline.

Prevalence of cross-lingual citations over time across Mathematics, Physics, and Computer Science.

Mathematics papers are notably more likely to cite non-English works—often twice as frequently as papers in Computer Science or Physics. This likely reflects the deep historical roots of mathematical research in non-English speaking regions, particularly the former Soviet Union.

2. The "Resolution Gap"

One of the most critical findings is the lack of visibility for non-English research. While 42.6% of all references in the dataset could be linked to unique identifiers in the Microsoft Academic Graph (MAG), only 11.2% of cross-lingual citations were resolvable. This "resolution gap" means the vast majority of non-English citations are effectively invisible to standard impact metrics, potentially undervaluing research published in other languages.

3. Consistency in Citation Intent

Contrary to expectations, authors do not cite non-English papers "differently." The distribution of citation intent—whether for background information, methodology, or results—is nearly identical between cross-lingual and monolingual citations. This suggests that when authors cross language barriers, they do so for the same rigorous scientific reasons they cite any other work.

  • Russian remains the most cited non-English language (66% of the identified set), but citations to Chinese works are on a sharp upward trajectory.
  • Local Gatekeepers: 37% of cross-lingual citations are "local," where an author affiliated with a non-English speaking country cites a paper in their native language within an English-language publication. These authors serve as vital bridges between local research and the global discourse.

Conclusion and Future Directions

While cross-lingual citations are relatively rare, they represent essential connections in the global scientific network. The study highlights a significant technical challenge: current citation databases are poorly equipped to track and credit non-English research. Future work aims to improve automated language identification for bibliographies and expand the analysis to the humanities, where cross-lingual citation patterns are expected to be even more significant.

Summary updated

Abstract

Citation information in scholarly data is an important source of insight into the reception of publications and the scholarly discourse. Outcomes of citation analyses and the applicability of citation based machine learning approaches heavily depend on the completeness of such data. One particular shortcoming of scholarly data nowadays is that non-English publications are often not included in data sets, or that language metadata is not available. Because of this, citations between publications of differing languages (cross-lingual citations) have only been studied to a very limited degree. In this paper, we present an analysis of cross-lingual citations based on over one million English papers, spanning three scientific disciplines and a time span of three decades. Our investigation covers differences between cited languages and disciplines, trends over time, and the usage characteristics as well as impact of cross-lingual citations. Among our findings are an increasing rate of citations to publications written in Chinese, citations being primarily to local non-English languages, and consistency in citation intent between cross- and monolingual citations. To facilitate further research, we make our collected data and source code publicly available.
BibTeX
@article{saier2022cross,
  author = {Tarek Saier and Michael Färber and Tornike Tsereteli},
  title = {{Cross-Lingual Citations in English Papers: A Large-Scale Analysis of Prevalence, Usage, and Impact}},
  journal = {Int. J. Digit. Libr.},
  year = {2022},
  url = {https://openreview.net/forum?id=qgqPc2cm_1},
}
Figure