Introduction
In the field of Statistical Machine Translation (SMT), word alignment is a critical step that identifies correspondences between words in source and target languages. Standard practice involves training two directional models—source-to-target ($e \to f$) and target-to-source ($f \to e$)—and subsequently merging them through a process known as symmetrization.
Traditional symmetrization techniques, such as the widely used "Grow-Diag-Final" (GDF) heuristic, are often ad-hoc and lack a solid mathematical foundation. These methods typically struggle to generalize across diverse language pairs, lack tunable parameters for specific dataset optimization, and do not scale effectively when combining more than two alignment models. This paper addresses these limitations by introducing a principled framework based on the power mean.
The Power Mean Framework
The authors propose using the generalized mean, or power mean, to merge alignment links. The power mean of a set of values $a_1, \dots, a_n$ is defined as:
$$S_p(a_1, \dots, a_n) = \left( \frac{1}{n} \sum_{k=1}^n a_k^p \right)^{1/p}$$
The strength of this approach lies in the parameter $p$, which allows the algorithm to traverse a continuous spectrum of combination strategies. As $p \to 0$, the function converges to the geometric mean, effectively representing the intersection of alignment tables (optimizing for precision). Conversely, as $p \to \infty$, it becomes the maximum function, representing the union (optimizing for recall).
Optimization and Implementation
Rather than relying on fixed heuristics, the parameter $p$ is treated as a tunable variable. The authors utilize the Nelder-Mead simplex algorithm to optimize $p$ against the F-measure of a small "gold standard" hand-aligned dataset. Furthermore, the framework supports weighted combinations, allowing the system to prioritize more reliable alignment models. Because the power mean generates "soft" scores, a secondary binarization step based on link adjacency is employed to produce the final discrete alignment links.
Experimental Setup
The proposed method was evaluated on an English-Pashto translation task using approximately 100,000 parallel sentences for training. The development set included 2,026 sentences, with a 150-sentence subset used for tuning $p$. The researchers combined five distinct alignment models, including GIZA++, the Berkeley Aligner, and specialized HMM-based and dependency-aware models.
Key Results
The experimental results demonstrate that the power mean approach ($PM_n$) significantly outperforms traditional heuristics. By combining five alignment tables, the $PM_n$ method achieved an F-measure of 0.7276, surpassing both GDF (0.6712) and simple Union (0.6589).
Crucially, these improvements in alignment quality translated directly into better translation performance. In the English-to-Pashto direction, $PM_n$ reached a BLEU score of 0.1209, a notable improvement over the GDF baseline of 0.1091.
Beyond quality metrics, the $PM_n$ approach proved more efficient. It produced phrase tables that were approximately 7% smaller than those generated by GDF, indicating that the method identifies more precise and useful alignment links rather than simply increasing the volume of links.
Conclusion and Future Work
The power mean framework provides a robust, scalable, and mathematically grounded alternative to ad-hoc alignment heuristics. By treating symmetrization as a continuous optimization problem, it allows for the effective combination of an arbitrary number of alignment models.
Future research will focus on bridging the "binarization gap" by integrating binary constraints directly into the optimization process. Additionally, the authors aim to explore optimizing $p$ directly for BLEU scores and extending the framework to accept "soft" alignment inputs, such as posterior probabilities, to further enhance translation accuracy.
Summary updated