Abstract
As global digital infrastructure increasingly relies on automated agents to relay information across borders, understanding the stability of sequential translation chains becomes critical. This study investigates the degradation of semantic meaning and text size when a source text is subjected to up to 284 consecutive translation "hops" using Google Translate. We propose an Accuracy Empirical Law (AEL) based on a power law decay model and demonstrate that translation quality is heavily influenced by the linguistic distance between languages in the chain.
Introduction
In the classic game of "Chinese Whispers," a message passed through a sequence of intermediaries inevitably morphs into something unrecognizable. In the context of Neural Machine Translation (NMT), a similar phenomenon occurs. While NMT has achieved high performance for single-hop translations, the holistic behavior of translation chains—where text moves from Language A to B, then B to C, and so on—remains poorly understood. This paper quantifies this degradation, seeking to determine if there is a mathematical regularity to how errors compound and how linguistic ancestry affects this decay.
Methodology
The researchers utilized the Google Translate API to construct three types of translation chains:
- Randomized: 71 languages selected from Google’s 103 available languages, ordered randomly.
- Common: Languages with close common ancestry (e.g., the Neolatine group: Italian, Portuguese, Spanish).
- Mixed: Languages from widely divergent families (e.g., Slavic, Sino-Tibetan, Germanic).
The primary metric used was the GLEU (Google Bilingual Evaluation Understudy) score, which measures the minimum of recall and precision of n-grams between the output and the original source. Four seed texts were used: two short excerpts (t1, t3) and two longer excerpts (t2, t4) in English and Portuguese.
The Accuracy Empirical Law (AEL)
The study's central finding is that accuracy loss follows a predictable power law: $$AEL(t) = (t + 1)^{-\alpha}$$ where $t$ represents the number of translation hops and $\alpha$ is the semantic divergence factor. The model fit the empirical data with a remarkably low Root Mean Square Error (RMSE) of 0.02 to 0.04.
The divergence factor $\alpha$ varies significantly based on linguistic proximity. For "common" language groups, $\alpha$ was measured at 0.110, whereas for "mixed" groups, it increased to 0.290—nearly triple the rate of decay.
The "Textual Black Hole" Phenomenon
Beyond semantic accuracy, the researchers observed a consistent reduction in text size, a phenomenon they termed the "textual black hole." As translations progress, the AI tends toward brevity through fragmentation (omission of connectors), incoherent syntax, and literalism.
In one extreme case, a 258-word text was reduced to just seven words after 284 random hops: "Read the Bible and forget it." This suggests that NMT systems have an inherent bias toward summarization when faced with compounding uncertainty.
Conclusion
The discovery of the Accuracy Empirical Law provides a mathematical framework for predicting information loss in automated relay systems. The results underscore the importance of linguistic genealogy; translating within the same language family preserves meaning significantly longer than jumping between unrelated families. Future work may involve applying this law to optimize translation paths, potentially identifying "bridge" languages that minimize semantic divergence.
Reference: Sequeira, L. N., Moreschi, B. S., Cozman, F. G., & Fontes, B. (2020). An Empirical Accuracy Law for Sequential Machine Translation: the Case of Google Translate. arXiv:2003.02817v2 [cs.CL].
Summary updated