Introduction
Code-mixing—the practice of alternating between two or more languages in a single conversation—is a ubiquitous phenomenon in bilingual communities, resulting in hybrid dialects like "Hinglish," "Spanglish," or "Benglish." Despite its prevalence, machine translation (MT) systems frequently struggle with code-mixed text due to a severe lack of "gold standard" parallel data, where a code-mixed sentence is paired with its monolingual equivalent. This "data desert" has historically forced researchers to rely on complex synthetic data generation pipelines.
Previous solutions often required a suite of external, language-specific tools, including language identifiers, word aligners, and transliterators. These dependencies make systems brittle, difficult to maintain, and challenging to scale to low-resource languages that lack such specialized linguistic infrastructure. To address these limitations, researchers from IIT Bombay and Google Research have introduced Back-to-Back Translation (B2BT), a streamlined two-stage training scheme that leverages the internal capabilities of multilingual models to bridge the data gap.
The B2BT Methodology
The core innovation of B2BT is its ability to use a multilingual model as its own teacher, eliminating the need for external alignment tools. The process unfolds in three primary phases:
- Base Model Training: The process begins with a multilingual model ($M$) trained on standard parallel data (e.g., pure Hindi to English) and non-parallel code-mixed data.
- Stage 1: Fine-tuning for $E \to C$: To enable the model to generate code-mixed (CM) text, the researchers take a monolingual CM corpus and translate it "backwards" into English using the base model. The model is then fine-tuned on these synthetic English-to-CM pairs, resulting in an intermediate model ($M'$) capable of producing realistic code-mixed output.
- Stage 2: Fine-tuning for $C \to E$: High-quality in-domain English sentences are translated into code-mixed sentences using $M'$. This creates a high-quality synthetic CM-to-English parallel dataset. The final model ($M^*$) is fine-tuned on this data to achieve superior translation performance.
The Source-Side Masking Trick
A critical technical detail in B2BT is the application of source-side masking during fine-tuning. Because code-mixed sentences often contain English words identical to the target English words, models can fall into a "lazy" copy-paste strategy. By masking 20% of the source tokens, the authors force the model to learn the deeper semantic relationships between the matrix language (e.g., Hindi) and the target language (English).
Experimental Setup and Datasets
The study evaluated B2BT across four language pairs, all translating into English:
- Hindi (HiEn): Utilizing the IIT Bombay English-Hindi corpus.
- Bengali (BnEn): Utilizing the Samanantar dataset.
- Spanish (EsEn): Evaluated on the LinCE Machine Translation leaderboard.
- Marathi (MrEn): A new dataset introduced by the authors, sourced from the Spoken Tutorial Project.
Training data incorporated parallel corpora from WMT and Samanantar, alongside monolingual code-mixed data harvested from social media and educational transcripts.
Experimental Results
B2BT demonstrated significant performance gains over existing benchmarks. It outperformed the previous state-of-the-art method (mBertAln) by +3.8 BLEU on Hindi-English and +2.8 BLEU on Bengali-English. On the LinCE leaderboard for Spanish-English, B2BT achieved the top score, surpassing the best baseline by +6.5 BLEU.
Beyond automated metrics, human evaluation confirmed the quality of the approach. Human raters preferred B2BT’s synthetic data over alignment-based methods in 39% of cases, compared to only 17% for the older methods. The model proved particularly robust on "hard" subsets of data, such as sentences with high degrees of code-mixing or numerous out-of-vocabulary words.
Conclusion and Future Work
B2BT demonstrates that a well-trained multilingual model already possesses sufficient linguistic knowledge to act as its own teacher through iterative back-translation. By exploiting the inherent asymmetry of code-mixed translation—where translating from CM to English is easier due to shared grammatical structures—B2BT achieves state-of-the-art results without external linguistic tools.
While the method still requires a corpus of monolingual code-mixed text, its success suggests a path toward integrating code-mixed support into massively multilingual frameworks like NLLB or M2M-100, potentially providing out-of-the-box translation for hundreds of hybrid dialects worldwide.
Summary updated