TL;DR
This paper tackles the challenge of controlling formality in machine translation, especially for low-resource language pairs like English-Hindi and English-Japanese, where labeled data is scarce. They propose a novel approach combining rule-based and model-based post-editing strategies with a data augmentation technique that leverages a formality classifier trained on weakly-labeled data, achieving significant improvements in formality accuracy while maintaining translation quality.
What Problem Does This Paper Address?
Neural Machine Translation (NMT) models have achieved remarkable fluency, but they often struggle with controlling stylistic attributes like formality. While generic parallel corpora are abundant, they implicitly assume a single "correct" translation, neglecting the nuances of language style. This is particularly problematic for formality control, where specific linguistic choices can convey politeness, social distance, or courtesy. The IWSLT 2022 formality control task highlights this challenge, especially for language pairs like English-Hindi (en→hi) and English-Japanese (en→ja), where very limited formality-annotated data is available for training. This data sparsity makes it difficult to fine-tune NMT models to reliably produce translations in a desired formality level.
Methods
The authors propose a two-pronged approach to improve formality control:
- Post-Editing Strategies: After an initial translation is generated, these strategies aim to correct or alter its formality. They explore three methods:
- T-V Conversion: A language-specific rule-based approach for Hindi, leveraging the distinction between informal "T-form" and formal "V-form" pronouns and their corresponding verb conjugations.
- Verb Conjugation: A language-specific approach for Japanese, where verb morphology is used to signal formality. This involves using a morphological analyzer to identify verb classes and then applying dictionary rules.
- Sequence-to-Sequence Model: A language-agnostic approach using a Transformer-based pointer network. This model is trained to rewrite informal text to formal, or vice-versa, by learning to copy parts of the input while generating the output. It's pre-trained as an auto-encoder and then fine-tuned for formality transfer.
- Weakly-Labeled Data Augmentation: To combat data sparsity, they develop a method to automatically generate more formality-labeled data.
- Formality Classifier: They train a multilingual text formality classifier using a small set of "seed" formality-annotated data. This seed data is generated using rule-based annotations (T-V conversion for Hindi, formality-indicating verbs for Japanese).
- Automatic Labeling: The trained classifier is then used to label a large unannotated parallel corpus. Only segments with high prediction confidence (above 0.95) are selected as augmented data. This augmented data, along with the original IWSLT training data, is used to fine-tune the NMT model.
Datasets
- Training Data:
- Generic NMT Pre-training: Large-scale parallel corpora were used, including the IIT Bombay English-Hindi parallel corpus for en→hi, and WikiMatrix and JParaCrawl for en→ja.
- Formality Control Fine-tuning: The IWSLT Formality dataset was used.
- Data Augmentation: Unannotated parallel corpora were leveraged, with seed data generated from linguistic cues (T-V distinction for Hindi, formality-indicating verbs for Japanese).
- Monolingual Data: Used for pre-training the sequence-to-sequence post-editing model.
- Evaluation Data: The novel IWSLT Formality Dataset from Nădejde et al. (2022) was used, which provides source segments with contrastive reference translations for both informal and formal styles.
Key Insights and Ideas
- Language-Specific Strategies Shine: The paper strongly suggests that language-specific post-editing strategies (T-V conversion for Hindi, verb conjugation for Japanese) are more effective than a purely language-agnostic seq2seq model for formality control, especially when dealing with limited data.
- Data Augmentation is Crucial for Low-Resource Pairs: For the en→ja pair, the data augmentation strategy significantly boosted formality accuracy, demonstrating its value in data-scarce scenarios.
- Formality Classifier Quality Matters: The effectiveness of data augmentation for en→hi was limited, potentially due to the noisier seed data generated by a simpler heuristic (T-V pronoun matching) compared to the more robust method used for Japanese.
- Post-Editing is a Powerful Add-on: Even without data augmentation, rule-based post-editing strategies significantly improved formality accuracy over just fine-tuning the NMT model, while maintaining comparable BLEU scores.
- Directionality in Post-Editing: Uni-directional post-editing (e.g., formal to formal, informal to informal) performed better than cross-directional edits (formal to informal), likely due to the precision-recall trade-offs of rule-based systems.
Key Results
- Formality Accuracy Improvements:
- For en→ja, formal accuracy improved from 93.9% to 96.2%, and informal accuracy from 98.1% to 100% with data augmentation.
- For en→hi, post-editing improved informal accuracy from 84.4% to 97.8%.
- BLEU Score Maintenance: The proposed methods generally maintained BLEU scores on par with the baseline, indicating that formality control did not come at the cost of translation fluency.
- Submitted System Performance: The final submitted system for en→hi used finetuned mBART + data augmentation, while the en→ja system used finetuned mBART + data augmentation + verb conjugation post-editing. Table 5 shows the final performance on the blind submission.
Limitations and Future Work
- En→hi Data Augmentation Noise: The authors acknowledge that the data augmentation for en→hi might have been affected by noisy seed data, leading to degraded formality accuracy compared to the baseline. Future work could involve using more comprehensive heuristics for seed data generation.
- BLEU Score Degradation for En→hi: The proposed system for en→hi showed a slightly worse BLEU score than the finetuned mBART model. This is attributed to the augmented data coming from a different domain (conversational) than the test set. Future work could involve augmenting training data with more conversational data or up-sampling the IWSLT formality data.
- Seq2Seq Model for Formal to Informal: The seq2seq model was primarily used for formal-to-informal conversion, with the assumption that the hypothesis was already formal. Further exploration of its bidirectional capabilities could be beneficial.
Summary updated