Introduction
Statistical Machine Translation (SMT) from Chinese typically requires an initial word segmentation step, as Chinese text lacks explicit word boundaries. Conventional approaches rely on off-the-shelf Chinese Word Segmentation (CWS) tools trained on manually annotated corpora. However, these "linguistically correct" segmentations often fail to align with the optimal units for translation, particularly when dealing with out-of-vocabulary (OOV) words or domain-specific terminology.
This paper presents a Bayesian semi-supervised model that learns to segment Chinese text specifically to maximize translational utility. By leveraging both monolingual patterns and bilingual alignment data, the proposed method discovers a task-optimal lexicon that improves downstream SMT performance.
Methodology
The core of the approach is a generative model based on a Unigram Dirichlet Process (DP).
Monolingual Word Model
The model treats word generation as a Chinese Restaurant Process (CRP). In this framework, each word is viewed as a "customer" sitting at a "table" representing a word type. This naturally balances the use of frequent, known words with the discovery of new words, controlled by a concentration parameter $\alpha$. A Poisson-based spelling model ensures that newly discovered words maintain statistically plausible lengths.
Bilingual Integration
To ensure segmentations are MT-friendly, the authors incorporate bilingual information via a log-linear framework. This includes an "inverse" IBM Model 1, which encourages segmentations that map consistently to English words. The model considers both Chinese-to-English and English-to-Chinese directions to capture robust alignment features.
Inference via Gibbs Sampling
Given the complexity of the joint segmentation and alignment space, the authors employ Gibbs sampling. This Markov Chain Monte Carlo (MCMC) method iteratively re-samples word boundaries and alignments, allowing the model to converge on an effective segmentation without requiring an exhaustive search.
Experimental Setup
The model was evaluated on two tracks:
- NIST 2005 (Large Track): A large-scale news dataset with ~2 million parallel sentences. Due to computational constraints, only the monolingual Bayesian model was applied.
- IWSLT 2007 (Small Track): A travel-domain dataset with ~43,000 sentences, utilizing the full bilingual model.
Results and Analysis
Large-Scale Performance (NIST)
On the NIST datasets, the Bayesian model (GS) consistently outperformed the standard LDC baseline. As shown in Figure 3, the method achieved absolute BLEU score improvements of up to 0.41% (NIST 2005). This suggests that a flexible, data-driven lexicon is superior to a fixed manual one, even in high-resource scenarios.
Small-Scale Performance (IWSLT)
In the IWSLT track, the full bilingual model significantly outperformed established segmenters like ICTCLAS and the LDC tool. On the evaluation set, the Bayesian method reduced the Word Error Rate (WER) by approximately 1.1% compared to the baseline and improved BLEU scores (see Figure 5). Notably, the model achieved a WER of 45.9, matching the best-performing ICT baseline while providing a more principled framework for OOV handling.
Discussion and Conclusion
The results highlight that the "best" segmentation for SMT is task-dependent rather than purely linguistic. By using a Dirichlet Process prior, the model successfully discovers new words not present in initial dictionaries. Furthermore, bilingual data acts as a powerful disambiguator; characters that consistently map to a single English unit are likely to be merged into a single Chinese word.
While effective, the primary limitation remains the computational cost of Gibbs sampling, which restricted the use of the full bilingual model on the largest datasets. Future work involves exploring more efficient inference techniques and more complex alignment models beyond IBM Model 1 to further refine the segmentation process.
Summary updated