Replacing Discrete Identifiers with Semantic Text Embeddings

33 concepts25 papers1 proposal

For decades, recommendation engines and intelligent tutoring systems have relied on a common abstraction: the discrete identifier. Whether it is a product ID in an e-commerce catalog (Item_48291), a movie ID on a streaming platform, or a concept ID in an educational curriculum, systems represent items as unique, atomic tokens.

To make these tokens usable by machine learning models, they are typically mapped to randomly initialized vector embeddings. Through massive amounts of interaction data—such as user clicks, purchases, or test scores—the model gradually updates these vectors so that items with similar interaction patterns sit close together in a latent space.

While highly effective for closed systems with stable catalogs, this ID-centric paradigm suffers from fundamental structural limitations. Because discrete IDs are inherently orthogonal and carry no intrinsic meaning, they create a severe cold-start problem: a brand-new item or question has no interaction history, leaving its embedding completely uninformative. Furthermore, ID spaces are isolated silos. An embedding learned for a book on one platform cannot easily transfer to a movie on another, trapping models within domain boundaries.

A rapidly growing body of research addresses these limitations by replacing discrete identifiers with semantic text representations. By leveraging pre-trained language models (LMs) to encode natural language descriptions, titles, or metadata directly, these systems map items into a universal, continuous semantic space.

Why Text-Based Representations Work

To understand why text-centric representations generalize so effectively, consider how a traditional model learns that two items are related. If a system wants to recommend Interstellar to someone who liked Inception, it must observe thousands of users interacting with both distinct IDs.

In contrast, a text-based model processes the raw textual metadata of the items. A pre-trained language model already possesses rich world knowledge and linguistic understanding; it inherently knows that "directed by Christopher Nolan" and "sci-fi space exploration" share deep thematic and conceptual links.

By passing item descriptions through a text encoder, the system bypasses the need to learn item relationships entirely from scratch. As Mu et al. demonstrate in ID-Agnostic User Behavior Pre-training for Sequential Recommendation, representing items through text descriptions allows sequential recommendation models to capture underlying user preferences for specific product attributes like brand, material, or style. This bridges the semantic gap inherent in traditional systems where abstract IDs fail to convey the actual characteristics of a product.

Similarly, Halder et al. show in POI recommendation with queuing time and user interest awareness that rich textual descriptions of physical locations provide significantly more predictive power than broad categorical labels, allowing models to identify nuanced similarities between venues based on their specific atmospheres or features.

Breaking the Domain Barrier

Because language serves as a universal interface, text-centric item representations naturally enable cross-domain and cross-platform transfer. If two platforms describe their items in natural language, they share a vocabulary—even if their user bases and item catalogs are completely disjointed.

In Towards Universal Sequence Representation Learning for Recommender Systems, Hou et al. introduce UniSRec, a framework that leverages text descriptions and a specialized adapter to train a sequence representation model on one set of domains and immediately deploy it on entirely different platforms with zero overlapping users or items. This concept is extended to multi-modal contexts by Du et al. in Multi-Modality is All You Need for Transferable Recommender Systems, where mapping items to a shared semantic space using text and image encoders allows a model to transfer behavioral knowledge from e-commerce shopping directly to short-video browsing.

This shared linguistic space resolves what researchers call the "overlap dilemma"—the traditional requirement that cross-domain recommendation systems need a critical mass of shared users to align different ID spaces. As Liu et al. demonstrate in Bridge the Domains: Large Language Models Enhanced Cross-domain Sequential Recommendation, high-dimensional semantic embeddings derived from text attributes allow large language models to bridge disparate domains like books and movies in highly data-sparse scenarios.

In a similar vein, Guo et al., writing in Semantic-enhanced Co-attention Prompt Learning for Non-overlapping Cross-domain Recommendation, utilize text encodings to transfer "cluster-level" knowledge across non-overlapping domains, proving that a model can infer a user's cinematic tastes based on their literary preferences by mapping both into a unified semantic space. As summarized by Wu et al. in A survey on large language models for recommendation, shifting from ID-centric interaction matrices to text-centric representations transforms recommendation from a statistical co-occurrence problem into a knowledge-aware reasoning task.

From Retrieval to Generative Modeling

When items are represented as text, recommendation can be reframed from a traditional ranking or retrieval task into a generative natural language task. Instead of predicting a probability score over a fixed vocabulary of millions of item IDs, a language model can be trained to directly generate the textual identifier or title of the next item.

In GenRec: Large Language Model for Generative Recommendation, Ji et al. fine-tune a large language model to directly write out the natural language title of the next item a user will interact with. This allows the system to exploit pre-trained semantic knowledge about genres, themes, and cultural contexts to infer relationships between items.

To better align the internal world knowledge of LLMs with recommendation tasks, Tan et al. propose IDGenRec: LLM-RecSys Alignment with Textual ID Learning, which replaces arbitrary numerical indices with semantically rich, textual identifiers generated by a dedicated model, enabling zero-shot recommendations across entirely different platforms.

Traditional ID Paradigm:
[User History: ID_042, ID_819] ──> [Matrix Factorization] ──> Predict: ID_901 (Opaque)

Text-Centric Generative Paradigm:
[User History: "Inception", "Interstellar"] ──> [Language Model] ──> Generate: "The Martian" (Semantic)

This textual alignment is particularly powerful in conversational interfaces, where the boundary between dialogue and recommendation is often blurred. Zhang et al. demonstrate in A Unified Multi-task Learning Framework for Multi-goal Conversational Recommender Systems that expanding an LM's vocabulary to pair unique item identifiers with their semantic names allows the model to learn both distinct item identities and their conversational contexts simultaneously.

By treating the entire recommendation pipeline as a pure language processing task, Zhang et al. show in Parameter-Efficient Conversational Recommender System as a Language Processing Task that representing items through structured metadata strings rather than graph-based identifiers allows parameter-efficient models (using techniques like LoRA) to seamlessly align item features with conversational context in a single training stage.

Furthermore, this deep semantic understanding allows models to look beyond immediate user-item interactions. In Leveraging LLMs for Influence Path Planning in Proactive Recommendation, Wang et al. exploit the internal world knowledge of LLMs to map out "influence paths"—sequences of recommendations that smoothly transition a user from their current interest to a completely new target topic based on shared historical, thematic, or cultural contexts.

Inductive Knowledge Tracing in Education

The benefits of text-based item representations extend far beyond product recommendations; they are also revolutionizing adaptive education. In intelligent tutoring systems, "knowledge tracing" models predict whether a student will answer a question correctly based on their past performance. Traditionally, this required manually tagging questions with discrete skill or concept IDs.

By replacing these abstract IDs with the raw text of the educational items, knowledge tracing models can perform inductive reasoning on entirely unseen material. As Srivastava et al. demonstrate in Question Generation for Adaptive Education, analyzing linguistic and conceptual features within raw question text allows a model to accurately predict student performance on novel, dynamically generated language exercises.

Traditional Knowledge Tracing:
[Student History: Concept_12 (Pass), Concept_14 (Fail)] ──> Predict performance on Concept_12

Inductive Semantic Knowledge Tracing:
[Student History: "Solve for x: 2x+3=7" (Pass)] ──> Analyze Text ──> Predict performance on "Find x if 3x-1=5"

This approach completely removes the dependency on manual curriculum tagging. Liu et al. apply this to computer science education in Open-ended Knowledge Tracing for Computer Science Education, utilizing natural language embeddings of task descriptions to generalize performance predictions to novel coding problems.

To solve the cold-start problem for entirely new curricula, Fu et al. introduce SINKT in SINKT: A Structure-Aware Inductive Knowledge Tracing Model with Large Language Model. By replacing unique identifiers with semantic text embeddings and using an LLM to map structural relationships between concepts, the authors create a fully inductive knowledge tracing system capable of evaluating student mastery on brand-new questions by leveraging their linguistic and conceptual similarity to historical data.

Overcoming the Representation Bottleneck

If text-based representations are inherently superior for generalization, why haven't they completely replaced discrete IDs? The answer lies in a phenomenon known as anisotropy.

Standard pre-trained language models tend to compress text embeddings into a narrow, highly correlated cone within the vector space. In this anisotropic space, the embeddings of two completely different items might still have a high cosine similarity simply because of shared grammatical structures or common words. For collaborative filtering, which requires a highly discriminative space to distinguish subtle user preferences, this representation collapse degrades performance.

To address this bottleneck, researchers have introduced post-processing and alignment strategies. In Are ID Embeddings Necessary? Whitening Pre-trained Text Embeddings for Effective Sequential Recommendation, Zhang et al. prove that applying a statistical "whitening" transformation to pre-trained text embeddings removes anisotropy. By centering and rescaling the embedding space, they eliminate the need for discrete ID matrices entirely, allowing text-only models to outperform state-of-the-art hybrid models.

Furthermore, the scale of the underlying language model plays a critical role in the quality of the semantic space. In Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights, Li et al. explore the scaling laws of text-based collaborative filtering using models up to 175 billion parameters. Their findings reveal that while larger language models provide richer semantic representations that steadily improve recommendation capabilities, frozen embeddings alone are not a universal panacea; task-specific fine-tuning or trainable adapter layers remain essential to compress high-dimensional linguistic knowledge into optimal recommendation spaces.

Similarly, Shin et al. demonstrate in Scaling Law for Recommendation Models: Towards General-Purpose User Representations that when items are represented via natural language descriptions, user behavior modeling follows a predictable power-law scaling relationship. As data, compute, and model sizes scale up, the pre-training error drops predictably, yielding general-purpose user representations that can seamlessly bypass the vocabulary mismatch of heterogeneous item spaces.

By shifting the foundational representation of items from arbitrary tokens to natural language, the fields of recommendation and intelligent tutoring are moving away from isolated, data-hungry statistical models toward universal, cross-domain reasoning engines that understand why items are related through the shared interface of human language.

Key Foundational Work

Item-based Collaborative Filtering with BERT (2020) — contributed 2 concept elements to this direction.
Question Generation for Adaptive Education (2021) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_semantic_item_representation_a3a5624d,
  title = {semantic item representation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-item-representation/rdr_a3a5624db7574c6eac81daea4a0633b7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_semantic_item_representation_a3a5624d_v1,
  title = {semantic item representation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-item-representation/rdr_a3a5624db7574c6eac81daea4a0633b7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake