Abstract
This paper introduces SGNMT, an experimental platform for machine translation research. SGNMT provides a generic interface to neural and symbolic scoring modules (predictors) with left-to-right semantics, such as translation models (NMT), language models, translation lattices, $n$-best lists, or other kinds of scores and constraints. Predictors can be combined with other predictors to form complex decoding tasks. SGNMT implements a number of search strategies for traversing the space spanned by the predictors, which are appropriate for different predictor constellations. Adding new predictors or decoding strategies is particularly easy, making it a very efficient tool for prototyping new research ideas. SGNMT is actively being used by students and researchers at the University of Cambridge for coursework, theses, and core research.
Introduction
In standard Neural Machine Translation (NMT), the decoder is typically a hard-coded beam search algorithm tightly coupled to a specific neural architecture. This lack of flexibility poses a significant challenge for researchers wishing to introduce constraints—such as grammatical rules, external language models, or ensembling models with disparate tokenization schemes (e.g., BPE vs. character-level). Modifying existing codebases for such experiments often requires extensive "surgery."
SGNMT (Syntactically Guided Neural Machine Translation) addresses this by decoupling the scoring modules from the search algorithms. Developed at the University of Cambridge, it provides a modular "Lego-like" framework where everything that provides a score is treated as a generic "predictor."
System Architecture
The core of SGNMT's flexibility lies in two central abstractions: Predictors and Decoders.
- Predictors: These are scoring modules that provide a probability distribution over the target vocabulary given the current state. They implement a simple API consisting of
predict_next()to retrieve scores andconsume()to update internal state based on a selected token. - Decoders: These are search strategies that traverse the space defined by the predictors. While beam search is standard, SGNMT also supports A* search, Depth-First Search (DFS), and greedy decoding.
By standardizing these interfaces, SGNMT allows for the linear combination of multiple predictors to guide a single search process. This modularity enables the rapid prototyping of hybrid systems that combine the fluid generation of neural models with the rigid control of symbolic tools like Finite State Transducers (FSTs).
Versatile Predictors and Search Strategies
SGNMT supports a wide array of predictors, ranging from modern neural models to traditional statistical components.
Beyond simple combinations, SGNMT introduces technical innovations such as:
- FST Wrappers: These allow word-based NMT models to be used even when the search operates at the character or BPE level by "masking" invalid transitions.
- Synchronized Beam Search (
syncbeam): This strategy enables the ensembling of models with different tokenization schemes by synchronizing hypotheses at specific boundaries (e.g., word endings). - Batch Decoding: To mitigate Python's performance overhead, SGNMT utilizes a multi-threaded scheduler to group sentences into large GPU batches, significantly increasing throughput.
Evaluation and Results
The platform's utility has been demonstrated across several benchmarks, including WMT English-French and ASPEC Japanese-English tasks.
- Performance: Despite its Python implementation, SGNMT's batch decoder achieved a throughput of 911.6 words per second on a Titan X GPU. This performance is competitive with high-performance C++ decoders in sequential modes.
- Accuracy: Using the
sepbeamstrategy for system-level combination, researchers achieved a 0.5 BLEU gain on the Japanese-English ASPEC task by ensembling three different BPE-based NMT models. - Versatility: The framework has successfully powered diverse research projects, including sub-word modeling in SMT and even automatic music composition, where LSTMs provided melodic "creativity" while Weighted Finite State Acceptors (WFSAs) enforced musical constraints.
Conclusion
SGNMT is explicitly designed for research and prototyping rather than production-level latency. Its primary strength is the ease with which new models and search strategies can be integrated and tested. By providing a standardized interface for scoring and searching, SGNMT empowers researchers to move beyond standard beam search and explore complex, constrained decoding spaces. Future work focuses on expanding the library of predictors to include models from a wider variety of deep learning frameworks.
Summary updated