Abstract
Click-Through Rate (CTR) prediction in industrial recommendation systems faces a significant "memory wall" when scaling to billions of users and items. Traditional embedding-based methods require massive RAM, while content-encoding methods suffer from high latency. This paper introduces UIST (User-Item Semantic Tokenization), a paradigm that discretizes high-dimensional embeddings into hierarchical semantic tokens. By utilizing Residual Quantization (RQ-VAE) and a Hierarchical Mixture Inference (HMI) module, UIST achieves a 200-fold reduction in memory usage while maintaining nearly 99% of the accuracy of state-of-the-art models with industrial-grade inference speeds.
Introduction
In modern industrial recommendation systems, predicting the Click-Through Rate (CTR) is a balancing act between three competing factors: accuracy, latency, and memory consumption. As datasets grow to encompass billions of users and items, existing paradigms have begun to hit a performance ceiling:
- Content-Encoding Paradigm: This approach utilizes heavy pretrained models (such as BERT) to encode item content dynamically. While highly accurate, the computational overhead results in high latency, making it unsuitable for real-time applications.
- Embedding-Based Paradigm: This approach pre-calculates dense embedding vectors and caches them for fast retrieval. While this minimizes latency, storing 768-dimensional vectors for billions of entities creates a massive memory bottleneck, often requiring hundreds of gigabytes of RAM.
The authors of this study address this "memory wall" by proposing a method to incorporate rich semantic information from item text and user history without the prohibitive storage costs of dense floating-point embeddings.
The UIST Framework
The proposed framework, UIST (User-Item Semantic Tokenization), shifts the storage paradigm from raw floats to discrete "semantic IDs" or tokens. The framework is composed of three primary components:
1. Semantic Representation
To capture the underlying meaning of entities, UIST first extracts features using pretrained models for items. For users, it employs a transformer-based autoencoder to learn representations from historical behavior sequences. These representations serve as the input for the tokenization process.
2. Discrete Tokenization via RQ-VAE
Instead of storing a single large vector, UIST uses Residual Quantization (RQ-VAE) to discretize dense vectors into a short sequence of discrete tokens (e.g., a sequence of 4 tokens). These tokens are inherently hierarchical: the first token captures coarse, high-level semantic information, while subsequent tokens capture increasingly fine-grained details. This effectively creates a "semantic zip code" for every user and item.
3. Hierarchical Mixture Inference (HMI)
Because the tokens are hierarchical, simple interaction layers are insufficient to capture the relationship between users and items. The HMI module utilizes a base CTR model (such as DCN or DeepFM) to evaluate user-item token pairs at different levels of granularity. A linear layer then automatically weighs these coarse-to-fine interactions to produce the final click probability.
Experimental Evaluation
The researchers evaluated UIST using the MIND (Microsoft News Dataset), a standard benchmark for news recommendation containing over 65,000 items and 94,000 users. The evaluation focused on the trade-offs between compression, accuracy, and speed.
Massive Memory Compression
UIST achieved a ~200-fold reduction in space compared to the traditional embedding-based paradigm. In a simulated industrial environment, the memory requirements for storing user and item representations dropped from 572.20 GB to just 2.98 GB.
High Fidelity Accuracy
Despite the aggressive compression, UIST maintained 98% to 99% of the accuracy (measured by AUC and nDCG) of state-of-the-art embedding-based models. This suggests that discrete tokens can capture the vast majority of the predictive power contained in high-dimensional latent vectors.
Industrial-Grade Latency
UIST achieved an inference latency of 3ms, matching the speed of simple ID-based models. This is a significant improvement over content-encoding models, which often exceed 60ms of latency, making UIST viable for real-time production environments.
Discussion and Future Work
The success of UIST demonstrates that full-precision floating-point vectors are not strictly necessary for representing user interests or item categories. The hierarchical nature of the tokens, managed by the HMI module, proves that the structure of the quantization is a feature that the model can exploit to improve prediction quality.
While the current research focused on text-heavy news recommendations, the authors suggest that UIST could be extended to other domains, such as image-based or short-video recommendations. Furthermore, the sensitivity of the model to token length presents an opportunity for further optimization based on specific hardware constraints. Beyond CTR prediction, this tokenization approach offers a promising new perspective on general dataset compression for large-scale machine learning.
Summary updated