TL;DR
This paper tackles the challenge of building high-quality machine translation systems for Indian languages in the e-commerce domain, which is often characterized by low-resource data. They introduce a novel pipeline that leverages synthetic data generation, active learning, and domain adaptation to achieve colloquial and precise translations, outperforming existing public APIs.
What Problem Does This Paper Address?
The rapid growth of e-commerce platforms has led to a surge in users from diverse linguistic backgrounds, particularly in India. To cater to these users, product information, descriptions, and reviews need to be available in their native languages. However, building robust machine translation (MT) systems for many Indian languages is a significant challenge due to their "low-resource" nature. This means there's a scarcity of high-quality parallel data (text aligned in two languages) needed to train powerful neural MT models like Transformers. Furthermore, e-commerce content often requires colloquial and friendly language to resonate with users, a nuance that general-domain MT models struggle to capture. This paper aims to bridge this gap by developing large-scale, precise, and colloquial MT systems for Indian languages within the e-commerce context.
Methods
The core of the paper's contribution lies in their iterative training pipeline, which combines several key techniques:
- Synthetic Parallel Corpus Generation: Recognizing the lack of readily available parallel data, the authors generate synthetic parallel corpora. They achieve this by using monolingual data (which is abundant) and employing back-translation and forward translation methods.
- Iterative Model Training with Active Learning: The pipeline is designed to be iterative. They start with existing parallel corpora and then progressively incorporate more synthetic data. Crucially, they integrate an Active Learning component. This involves using a translation quality estimation module to identify potentially problematic translations, which are then sent to human annotators for correction. This feedback loop helps refine the models and improve the quality of synthetic data generation in subsequent iterations.
- Domain Adaptation: To ensure colloquial translations, the models are fine-tuned on in-domain e-commerce data. This step is vital for bridging the domain gap between general language corpora and the specific language used in product descriptions and reviews.
- Monolingual Data Cleaning: Before generating synthetic data, they employ a BERT-based classifier to detect and filter out noisy English texts from monolingual corpora. They also incorporate a spell correction module to further clean the data.
Datasets
The paper utilizes a combination of datasets:
- Publicly Available Parallel Corpora: Used as an initial seed for training.
- In-House Parallel Corpora: Collected from e-commerce specific data like Product Descriptions, Catalog Attributes, Search, and Product Reviews.
- Publicly Available Monolingual Corpora: From various sources (e.g., CCNet, OSCAR, news-crawl, Wikipedia) for synthetic data generation.
- Monolingual E-commerce Data: Collected internally from product descriptions, catalog attributes, and reviews.
They do not introduce a new benchmark dataset but rather focus on creating and utilizing synthetic data for existing low-resource languages.
Key Insights and Ideas
- Synthetic Data is Key for Low-Resource Languages: The paper strongly demonstrates that generating high-quality synthetic parallel data is a viable and effective strategy for overcoming the low-resource challenge in MT.
- Active Learning is Crucial for Iterative Improvement: The iterative nature of their pipeline, powered by active learning, allows for continuous refinement and significant improvements in translation quality over multiple iterations.
- Domain Adaptation is Essential for Colloquialism: Simply having large amounts of data isn't enough; fine-tuning on in-domain e-commerce data is critical for achieving the desired colloquial and user-friendly translations.
- Quality Estimation is a Powerful Filtering Tool: Robust translation quality estimation methods are vital for filtering out imperfect synthetic translations, which can otherwise degrade model performance.
Key Results
- Significant BLEU Score Improvements: The English to Hindi translation model saw a +9.02 BLEU point improvement over iterations. The best model achieved a BLEU score of 41.32, a substantial increase from the baseline of 32.3.
- Outperforming Public APIs: Their models consistently outperformed public translation APIs (like Google) on various test sets for Product Descriptions and Catalog Attributes across multiple Indian languages.
- High Translation Accuracy: The models achieved very high "Translation Accuracy" (defined as the rate of acceptable translations with minor errors), indicating their suitability for large-scale deployment. For English to Hindi, they reported >50% "gold standard" translations in manual evaluations.
- Deployment and Business Impact: The systems are deployed in batch-prediction mode and have led to a +11 bps improvement in conversion and significant cost savings through automated translations.
Limitations and Future Work
- Reliance on Synthetic Data: The pipeline heavily relies on synthetic translations. For extremely low-resource languages (e.g., Assamese with <1M monolingual text), the initial model might not be trained effectively, rendering the pipeline less effective. Data efficiency remains a challenge.
- Pipeline Complexity: The pipeline incorporates several language model-based sub-modules for data cleaning, quality estimation, etc., which can make it cumbersome to manage and update.
- Future Work: The paper implies that further improvements can be made by exploring more advanced data augmentation techniques, refining the active learning strategies, and potentially developing more efficient ways to handle extremely low-resource scenarios.
Summary updated