A Framework for Translating SMS Messages

Vivek Kumar Rangarajan Sridhar, John Chen, Srinivas Bangalore, Ron ShachamCOLING 2014

TL;DR

This paper presents a novel framework for translating SMS messages, tackling the challenges of noisy text and limited domain-specific data. They decouple the problem into unsupervised normalization and statistical machine translation, achieving good results on a user trial.

What Problem Does This Paper Address?

Short Message Service (SMS) is a ubiquitous communication tool, but its inherent brevity, use of abbreviations, and spelling errors make it difficult to translate accurately using standard machine translation (MT) techniques. Traditional MT systems rely on clean, well-formed text and large parallel corpora, which are scarce for the SMS domain. Existing work often focuses on normalizing SMS text to standard language, but this paper aims to directly translate SMS messages between languages, bridging the gap for cross-lingual communication.

Methods

The authors propose a two-stage approach:

  1. Unsupervised SMS Normalization: Instead of relying on manually created dictionaries or supervised learning, they leverage distributed word representations (learned via deep neural networks on a large Twitter corpus) to automatically discover mappings between noisy SMS forms and their canonical equivalents. This method learns both one-to-one mappings and can handle spelling errors.
  2. Hybrid Machine Translation: They employ a phrase-based statistical machine translation (SMT) system. To handle the repetitive nature of SMS, they incorporate a "cached table" mechanism that directly stores translations for common phrases, bypassing the statistical model for faster and potentially more accurate translations. They also use an automatic punctuation classifier to segment SMS messages into meaningful phrases before translation.

Datasets

The paper utilizes several data sources:

  • Speech-based SMS: Transcriptions of speech-based SMS collected via a smartphone app (41.3 million English, 2.4 million Spanish). A subset was manually translated to create a 50,000 parallel sentence corpus.
  • Amazon Mechanical Turk: A task was posted to collect SMS-like messages and their corresponding responses, resulting in 1000 English-Spanish parallel messages with shorthands.
  • Twitter: A large corpus of tweets was collected and filtered to create SMS-like messages. After manual translation, 6790 unique parallel sentences were obtained.
  • General Bitext: Large corpora of general parallel text (OpenSubtitles, Europarl, TED talks) were used for training the core SMT models.

Key Insights and Ideas

  • Unsupervised normalization is viable: The paper demonstrates that learning SMS normalization rules without explicit supervision is achievable and effective, leveraging the power of distributional semantics.
  • Decoupling is beneficial: Separating normalization from translation simplifies the problem and allows for specialized techniques for each stage.
  • Hybrid MT for SMS: Combining statistical MT with a cached table for common phrases is a practical approach to improve efficiency and accuracy for the repetitive nature of SMS communication.
  • Data augmentation is crucial: The use of multiple surrogate data sources (speech, MTurk, Twitter) was essential to build a sufficiently large corpus for training.

Key Results

  • The proposed framework achieved BLEU scores of 31.25 for English-Spanish and 37.19 for Spanish-English on a test set of real SMS interactions.
  • A user evaluation indicated that the majority of participants found the translation quality to be sufficiently adequate, with fluency rated as "good" to "non-native."
  • The unsupervised normalization scheme learned SMS shorthands with high precision and near-perfect recall.
  • Phrase segmentation was shown to significantly improve translation quality, with BLEU scores for English-Spanish increasing from 29.65 to 31.25 and for Spanish-English from 23.95 to 37.19 when segmentation was used.

Limitations and Future Work

  • One-to-one normalization: The unsupervised normalization currently only learns one-to-one mappings. Future work could explore learning many-to-one and many-to-many mappings for more complex shorthands.
  • Limited test set shorthands: The unsupervised normalization did not yield significant BLEU improvements on the test set because it contained only a few instances of shorthands.
  • User evaluation balance: The user study had a significant imbalance between English (20) and Spanish (5) participants, which affected the rate of message exchange.
  • Future directions: The authors suggest exploring confusion networks for normalization and investigating log-linear models for learning word representations. They also plan to continue harvesting data using the Amazon Mechanical Turk approach.

Summary updated

BibTeX
@inproceedings{sridhar2014framework,
  author = {Vivek Kumar Rangarajan Sridhar and John Chen and Srinivas Bangalore and Ron Shacham},
  title = {{A Framework for Translating SMS Messages}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2014},
}
Figure