Abstract: Pidong Wang, Hwee Tou Ng. Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
2013.
Introduction
Natural Language Processing (NLP) systems, particularly Machine Translation (MT), are typically optimized for formal text such as news reports or legal documents. However, the rise of social media platforms like Twitter and Weibo has introduced a significant challenge: "noisy" text. Informal communication often features non-standard abbreviations, omitted punctuation, and grammatical shortcuts. These deviations cause a sharp decline in performance for downstream tasks; for instance, part-of-speech tagging accuracy can drop from 90% on news text to 76% on social media content.
While previous research has largely focused on word-level substitutions (e.g., replacing "u" with "you"), this paper argues that effective normalization requires a holistic approach that addresses structural issues like missing pronouns and punctuation. The authors propose a beam-search decoder framework designed to transform informal social media posts into formal language suitable for standard MT systems.
The Beam-Search Framework
Rather than relying on a simple rule-based pipeline, the proposed system treats normalization as a search problem. The goal is to find the most probable formal sequence given an informal input string.
The framework utilizes several key components:
- Hypothesis Producers: Modular engines that suggest potential transformations, including dictionary lookups, pronunciation-based similarity (Pinyin for Chinese, phone-mapping for English), and rules for abbreviations.
- Structural Correction: A two-layer Dynamic Conditional Random Field (DCRF) model predicts punctuation and sentence boundaries, while a separate CRF model identifies and restores missing words (e.g., "be" verbs in English or dropped pronouns in Chinese).
- Scoring and Tuning: A linear model evaluates hypotheses by combining language model scores with various penalty and reward features. The weights for these features are optimized using Pairwise Ranking Optimization (PRO).
To ensure linguistic coherence, the decoder employs a contextual filter that checks proposed formal words against a 4-gram language model derived from a large formal corpus.
Data and Evaluation
The study introduced two new manually annotated corpora to facilitate research in this area:
- Chinese-English (CN2EN): 1,000 Weibo messages normalized and translated into formal English.
- English-Chinese (EN2CN): 2,000 SMS messages from the NUS SMS corpus, normalized and translated into formal Chinese.
These datasets allow for the evaluation of both normalization quality and the subsequent impact on machine translation performance.
Results and Discussion
The experimental results demonstrate that the beam-search approach significantly outperforms standard baselines, including phrase-based machine translation (PBMT) approaches to normalization.
Key findings include:
- Normalization Gains: The decoder achieved a BLEU score improvement of 9.98% for Chinese and 7.35% for English over baseline methods.
- MT Performance: Feeding the normalized text into a standard MT system resulted in a quality boost of 1.38 BLEU points for Chinese-to-English and 1.35 BLEU points for English-to-Chinese translation.
- Structural Importance: The authors found that fixing punctuation and restoring missing words are not merely cosmetic; they provide essential structural cues that allow MT systems to correctly identify sentence boundaries and clause structures.
Furthermore, the system is highly efficient, processing messages at an average rate of 0.2 seconds per message, making it viable for real-time applications.
Conclusion
This research highlights normalization as a "force multiplier" for NLP. By investing in a high-quality normalization pre-processor, researchers can leverage existing, powerful MT systems trained on formal text without the need for massive, scarce social media training data. Future work may explore tighter integration between normalization and translation, such as passing normalization lattices to the MT engine to mitigate error propagation in the pipeline.
Summary updated