Introduction
In the fields of Natural Language Processing (NLP) and Automatic Speech Recognition (ASR), large vocabulary models often face a significant memory "tax." While the core logic of these models resides in recurrent layers or transformers, the majority of the parameter budget is typically consumed by the "dictionary"—the massive embedding and softmax layers that map words to vectors and vice versa.
The Parameter Bottleneck
In a typical ASR system, the embedding (input) and softmax (output) layers can account for over 90% of the total parameters. This overhead is a critical barrier for on-device inference on mobile hardware with limited RAM and for federated learning, where transmitting massive updates is impractical.
The WEST Method
Google researchers introduced WEST (Word Encoded Sequence Transducers) to address this bottleneck. WEST is a single-stage compression technique that factors massive word-level matrices into the product of a structured sparse matrix ($C$) and a dense sub-unit matrix ($E^c$): $E = C \times E^c$.
By representing words as sequences of sub-units—such as characters or even random codes—WEST achieves up to 1000x compression of these layers. Crucially, the model maintains a word-level cycle, normalizing probabilities at the word step rather than the sub-unit step. This "MaxEnt" approach preserves the modeling benefits of word-level representations while utilizing the memory footprint of character-level models.
Reallocation and Performance
One of the most significant advantages of WEST is the ability to reallocate saved memory. By shrinking the "dictionary" layers, researchers can expand the "brains" of the model—the LSTM or recurrent layers.
In large-scale ASR tasks, this reallocation strategy allowed for a reduction in total model size from 15MB to 4.75MB. When combined with scalar quantization, the model reached 1.35MB (an 11x reduction) while actually improving the Word Error Rate (WER) by 3% relative to the baseline.
Conclusion
WEST demonstrates that linguistic characters are not the only way to compress models; random coding can be equally effective. By focusing on word-level normalization and strategic parameter reallocation, WEST enables the deployment of high-performance, large-vocabulary models on resource-constrained devices without sacrificing accuracy.
Reference: Variani, E., Suresh, A. T., & Weintraub, M. (2018). WEST: Word Encoded Sequence Transducers. arXiv:1811.08417v1 [cs.LG].
Summary updated