Abstract
The retrieval phase is a vital component in recommendation systems, requiring the model to be effective and efficient. Recently, generative retrieval has become an emerging paradigm for document retrieval, showing notable performance. These methods enjoy merits like being end-to-end differentiable, suggesting their viability in recommendation. However, these methods fall short in efficiency and effectiveness for large-scale recommendations. To obtain efficiency and effectiveness, this paper introduces a generative retrieval framework, namely SEATER, which learns SEmAntic Tree-structured item identifiERs via contrastive learning. Specifically, we employ an encoder-decoder model to extract user interests from historical behaviors and retrieve candidates via tree-structured item identifiers. SEATER devises a balanced k-ary tree structure of item identifiers, allocating semantic space to each token individually. This strategy maintains semantic consistency within the same level, while distinct levels correlate to varying semantic granularities. This structure also maintains consistent and fast inference speed for all items. Considering the tree structure, SEATER learns identifier tokens' semantics, hierarchical relationships, and inter-token dependencies. To achieve this, we incorporate two contrastive learning tasks with the generation task to optimize both the model and identifiers. The infoNCE loss aligns the token embeddings based on their hierarchical positions. The triplet loss ranks similar identifiers in desired orders. In this way, SEATER achieves both efficiency and effectiveness. Extensive experiments on three public datasets and an industrial dataset have demonstrated that SEATER outperforms state-of-the-art models significantly.
Introduction
In modern recommendation systems, the retrieval phase is a high-stakes first step: it must sift through millions of items to find a few hundred candidates in milliseconds. Traditionally, this is achieved using dual-encoders and Approximate Nearest Neighbor (ANN) search. Recently, Generative Retrieval (e.g., DSI or TIGER) has emerged as a promising alternative where a model is trained to directly "generate" the identifier (ID) of the next item.
However, existing generative methods often suffer from three major flaws:
- Semantic Blindness: They treat item IDs as arbitrary strings, ignoring the inherent hierarchical relationships (e.g., Category $\to$ Sub-category $\to$ Item).
- Structural Neglect: The tree structure used to create IDs is often ignored during the actual neural network training.
- Efficiency Bottlenecks: Imbalanced trees lead to inconsistent path lengths, creating unpredictable latency in production environments.
The SEATER Framework
To address these challenges, we introduce SEATER (SEmAntic Tree-structured item identifiERs), a framework that treats ID generation as a traversal through a structured semantic space.
Balanced k-ary Tree Construction
Instead of arbitrary identifiers, SEATER clusters items using constrained $k$-means based on pre-trained collaborative filtering embeddings (e.g., from SASRec). This creates a balanced tree where every item ID has the exact same length. This ensures semantic consistency: tokens at the same level of the ID represent the same level of granularity, and search depth remains logarithmic ($O(\log_k N)$), guaranteeing fast and stable inference.
Multi-Task Contrastive Learning
SEATER's core innovation lies in its training objectives. Beyond the standard Generation Loss, we incorporate two novel contrastive tasks to align the model with the tree structure:
- Alignment Loss ($\mathcal{L}_{ali}$): Using an infoNCE objective, this task ensures a "child" token's embedding is pulled closer to its "parent" token's embedding. This forces the model to internalize the hierarchical semantics of the identifiers.
- Ranking Loss ($\mathcal{L}_{rank}$): Using a triplet loss, this task teaches the model to distinguish between similar items. It rewards the model for ranking identifiers that share long prefixes with the ground truth higher than those that do not.
Experimental Evaluation
We evaluated SEATER on four diverse datasets: Yelp (dense, small-scale), Books (large-scale, sparse), News (MIND-small), and a massive industrial Micro-Video dataset (0.75M users, 85M interactions).
Performance and Efficiency
SEATER consistently outperformed state-of-the-art baselines, including dual-encoders like SASREC and generative models like TIGER. On the industrial Micro-Video dataset, SEATER achieved a 25.5% relative improvement in Recall@50 over the best baseline. Notably, SEATER achieves these results with a lightweight 1-layer Transformer architecture, making it highly viable for real-world deployment.
Ablation Studies
Ablation experiments confirmed the necessity of the proposed losses. Removing either the Alignment Loss or the Ranking Loss led to a significant performance drop, proving that explicitly training the model to understand the identifier structure is vital for effective generative retrieval.
Conclusion
SEATER demonstrates that item identifiers in generative retrieval should be treated as structured indices rather than simple text strings. By combining balanced tree construction with hierarchical contrastive learning, SEATER achieves a superior balance of retrieval accuracy and inference efficiency. Future work will explore handling "cold start" scenarios for new items and optimizing the branch number ($k$) for even larger item pools.
Summary updated