IDs That Mean Something: A Hands-On Tutorial on Semantic IDs for Recommender Systems

Roger Zhe Li, Aghiles Salah, Mete Sertkan, Changhong Jin, Yingjie Niu, Ruihai Dong, Barry SmytharXiv 2026 · DOI · arXiv

TL;DR

This paper proposes a comprehensive tutorial on Semantic IDs, a paradigm shift that replaces arbitrary integer identifiers in recommender systems with structured token sequences that encode actual item meaning. By embedding content, hierarchy, and behavioral signals directly into an item's identity, this approach solves long-standing issues like the "cold-start" problem and the "exploding embedding table" dilemma.

What Problem Does This Paper Address?

For decades, industrial recommender systems (RecSys) have relied on "raw IDs"—arbitrary integers like book #892 or movie #4200. While efficient for database lookups, these IDs are semantically hollow. They tell the model nothing about what the item actually is.

This leads to three major headaches:

  1. The Cold-Start Problem: When a new item enters the system with no interaction history, a raw ID-based model is essentially blind to it.
  2. Exploding Embedding Tables: As catalogs grow into the millions or billions, the memory required to store a unique embedding vector for every single raw ID becomes unsustainable.
  3. Fragmented Architectures: Incorporating "auxiliary information" (like text descriptions or images) usually requires adding complex, modality-specific sub-networks that are difficult to scale or transfer across different domains.

Methods

The authors advocate for Semantic IDs, which treat item identity as a structured, model-facing object. Instead of a single integer, an item is represented as a sequence of discrete tokens (e.g., [12, 45, 7]).

Key techniques for constructing these IDs include:

  • Vector Quantization: Using methods like RQ-VAE (Residual Quantized Variational AutoEncoder) to compress high-dimensional item content embeddings (from LLMs or vision models) into discrete codes.
  • Clustering: Applying algorithms like RQ-KMeans to group items hierarchically based on their features or user interaction patterns.
  • Fusion Strategies: The tutorial explores "early fusion" and "late fusion" methods to combine diverse data sources—text, images, and collaborative filtering signals—into a single, unified semantic identifier.

By using these methods, the "meaning" of the item is baked into the ID itself. Two items with similar semantic IDs are likely to be semantically similar, allowing the model to generalize across the catalog more effectively.

Datasets

As this is a tutorial proposal, the authors specify the datasets and tools they will use for the hands-on sessions:

  • Amazon Reviews Dataset: A standard benchmark in RecSys used for training and evaluating the semantic ID construction process.
  • Cornac: A specialized recommendation framework used to implement generative recommendation models and pre-compute baseline embeddings.

Key Insights and Ideas

  • Identity as a Structured Interface: The core "aha moment" is realizing that item identity doesn't have to be an opaque pointer; it can be a structured interface for sharing information across tasks, domains, and modalities.
  • Generative vs. Discriminative Unification: While semantic IDs are a natural fit for "Generative Recommendation" (where a model predicts the next ID in a sequence, similar to an LLM), they are also proving highly effective as structured input features for traditional "Discriminative Ranking" pipelines used by industry giants like Google, Meta, and Kuaishou.
  • The Design Tension: A critical insight is the balance required in ID construction. IDs must be "meaningful" enough to reflect content (for cold-start) but "recommendation-aware" enough to capture actual user behavior (for accuracy).
  • Native Cross-Domain Transfer: Because semantic structures can be shared across domains (e.g., "books" and "movies" might share high-level genre tokens), semantic IDs enable scalable cross-domain recommendation without the need for complex, explicit alignment models.
Table 1: Comparison with Related tutorials.

Key Results

While the paper itself is a tutorial proposal, it summarizes the quantitative and qualitative impact of semantic IDs in recent literature:

  • Scalability: Semantic IDs significantly reduce the size of embedding tables by allowing items to share tokens, making it possible to handle massive catalogs with much lower memory overhead.
  • Performance: Real-world deployments at companies like Meta and Google have shown that structured semantic representations improve ranking stability and performance, particularly for new or "long-tail" items.
  • Generalization: The tutorial highlights how semantic IDs provide a "principled alternative" for cross-domain transfer, allowing models to leverage knowledge from a source domain to a target domain more naturally.

Limitations and Future Work

The authors acknowledge several open challenges that define the next frontier for semantic IDs:

  • ID Stability: How do you update semantic IDs as a catalog evolves without forcing a complete retraining of the entire recommendation model?
  • Evaluation Metrics: Traditional metrics like "hit-rate" may not fully capture the benefits of semantic IDs; new evaluation frameworks are needed.
  • Semantic Planning: Moving beyond simple prediction to "semantic planning," where IDs help a model reason about long-term user, platform, and provider objectives.
  • Multi-Stakeholder Utility: Balancing the needs of different stakeholders (users, advertisers, and the platform) within the semantic ID structure.

Summary updated

Abstract

Traditional recommender systems rely on arbitrary integer IDs to identify items, which carry no semantic information and force models to learn representations from interaction data. While leveraging content information can partially mitigate cold-start, this paradigm inherently suffers from exploding embedding tables and scenario-specific architectures. Recent years have seen the emergence of semantic IDs, which encode semantic information directly into the item identity, reflecting content, hierarchy, and relationships. This framework addresses the above limitations by offering a principled way to encode large-scale semantic information into item identity, benefiting both generative and discriminative applications and unlocking more native unification across modalities, domains, and tasks. This tutorial traces the evolution from raw IDs to semantic IDs as a scalable means of information utilization, offering a comprehensive review of their development and applications across multi-modality, cold-start, and cross-domain recommendation. Hands-on exercises guide attendees through building and evaluating semantic IDs, training models, and exploring cross-domain transfer. We conclude by discussing open challenges, including dynamic semantic IDs and multi-stakeholder utility. All materials of the tutorial could be found at the official website https://semantic-id-tutorial.github.io.
BibTeX
@misc{li2026ids,
  author = {Roger Zhe Li and Aghiles Salah and Mete Sertkan and Changhong Jin and Yingjie Niu and Ruihai Dong and Barry Smyth},
  title = {{IDs That Mean Something: A Hands-On Tutorial on Semantic IDs for Recommender Systems}},
  year = {2026},
  eprint = {2611.04140},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.04140},
}
Figure