Introduction
Neural Machine Translation (NMT) has seen rapid advancements, yet experimenting with complex architectures—such as ensembling disparate models or enforcing symbolic constraints—often requires significant engineering effort. Standard decoding frameworks are frequently monolithic, tightly coupling the search algorithm with the underlying model. SGNMT (Syntactically Guided NMT) addresses this by providing an open-source platform that decouples translation models ("predictors") from search algorithms ("decoders"). This modularity enables researchers to rapidly prototype systems by mixing neural models with symbolic constraints like lattices or grammars.
Architecture: Decoupling Scoring from Search
The core innovation of SGNMT is its dual-concept architecture, which separates the "what" (scoring) from the "how" (searching). This separation is achieved through a standardized interface that allows any search strategy to interact with any scoring module.
Predictors (The "What")
Predictors are scoring modules that provide a probability distribution over the target vocabulary at each step. SGNMT defines a generic interface with methods such as initialize, predict_next, and consume. This abstraction allows the platform to wrap diverse models, from neural frameworks like TensorFlow and Theano/Blocks to symbolic models like Finite State Transducers (FSTs) and n-gram language models.
Decoders (The "How")
Decoders are the search algorithms that traverse the hypothesis space defined by the predictors. Because the interface is standardized, any decoder can work with any combination of predictors. SGNMT includes standard strategies like Beam Search and Greedy decoding, alongside specialized algorithms:
- A* Search: Uses heuristic functions defined via predictors to guide the search.
- Depth-First Search (DFS): Efficiently enumerates complete search spaces, which is particularly useful for exhaustive FST-based rescoring.
- Syncbeam: A specialized beam search that synchronizes models with different tokenization schemes (e.g., word-based vs. character-based) by comparing scores only when they reach a synchronization boundary.
Performance and Scalability
While SGNMT prioritizes flexibility, it remains competitive in terms of performance. To mitigate the overhead of its Python-based architecture, the platform introduces Batch Decoding. A multi-threaded scheduler groups sentences into large GPU jobs, significantly boosting throughput. On the WMT English-French task using a Titan X GPU, SGNMT achieved a decoding speed of 911.6 words per second. While specialized C++ engines like Marian-NMT are faster, SGNMT's speed is more than sufficient for research and prototyping.
Key Insights and Applications
SGNMT's strength lies in its ability to provide "Syntactic Guidance." By using symbolic constraints (like FSTs) to guide neural models, researchers can perform tasks like lattice rescoring or restricting output to specific vocabularies.
A notable feature is the handling of tokenization mismatches via the syncbeam decoder. This allows for ensembling models that operate on different subword units. Furthermore, the platform's flexibility has enabled creative applications, such as automatic music composition using LSTMs for creativity and Weighted Finite State Automata (WFSAs) to enforce musical constraints in the style of Bach chorales.
In practical evaluations, the platform demonstrated its utility in system-level combinations. By using the sepbeam strategy to combine three BPE-based NMT models, researchers reported a gain of 0.5 BLEU over a single system on the ASPEC Japanese-English task.
Conclusion
SGNMT serves as a bridge between high-performance production engines and the need for rapid research prototyping. By providing a framework-agnostic platform where scoring and search are strictly separated, it minimizes the implementation effort required for new ideas. It has proven particularly effective in academic settings, allowing students and researchers to implement complex projects without building decoders from scratch.
Summary updated