In sequential recommendation, the goal is to predict a user's next interaction based on their historical trajectory. Because behavioral data is often sparse—many users have only a handful of clicks—researchers frequently turn to Contrastive Learning (CL). The intuition behind CL is simple: take a user's sequence, create two slightly different "augmented views" of it, and train the model to recognize that both views belong to the same user.
However, a growing body of research suggests that the standard way we create these views is fundamentally flawed.
The Problem of Semantic Blindness
In computer vision, if you crop a photo of a cat, the result is still a cat. In sequential recommendation, the "semantics" are much more fragile. As Enneng Yang and colleagues point out in their survey of data augmentation, heuristic-based methods like random masking, cropping, or reordering are "semantically blind."
If a user's sequence is [iPhone, Case, Screen Protector], the core intent is clearly "buying a new phone." If a random mask removes iPhone, the sequence becomes [Case, Screen Protector], shifting the intent to "buying accessories." This phenomenon, known as semantic drift, forces the model to align two sequences that represent fundamentally different user goals. Yang Yu and Qi Liu argue in AdaptSSR that this leads to "contradictory training signals," where the model is punished for distinguishing between different intents simply because they originated from the same raw log.
Model-Level Augmentation: A Safer Path
To avoid corrupting the input data, some researchers suggest moving the "noise" from the data level to the model level.
Chong Liu and Xiaobo Liang introduced C2-Rec, which replaces complex data manipulations with simple dropout-induced noise. By passing the same sequence through a Transformer twice with different dropout masks, the model creates two views that are structurally identical but numerically different. This preserves the "semantic coherence" of the chronological behavior while providing the necessary variance for contrastive learning. Similarly, Ruihong Qiu uses this approach in DuoRec to solve "representation degeneration," ensuring that item embeddings don't collapse into a narrow cone without needing to delete critical items from the user's history.
Generative and LLM-Guided Refinement
If we must modify the data, the modification should be "intent-aware." Recent work has moved toward using generative models to ensure that augmented sequences remain plausible.
- Diffusion Models: Haolun Wu and Chen Ma developed CaDiRec, which uses a context-aware diffusion model to generate substitutions. Instead of randomly swapping an item, it generates a new item that fits the "logical flow" of the surrounding sequence.
- Large Language Models (LLMs): Bohao Wang and Jiawei Chen introduced LLM4DSR, which uses LLMs to identify "accidental clicks" or "clickbait" in a user's history. By substituting this noise with semantically relevant alternatives, they maintain the integrity of the sequence length and context. Lu Fan takes this further in CLLMRec, using LLMs to "prune" sequences based on semantic importance scores rather than random selection.
From Invariance to Equivariance
The most sophisticated evolution of this idea challenges the very premise of contrastive learning: that augmented views should be identical.
In ECL-SR, Peilin Zhou and Shoujin Wang argue for Equivariant Contrastive Learning. They suggest that while a model should be invariant to minor tweaks (like dropout), it should be equivariant to invasive changes (like item substitution). If you change an item in a sequence, the model's representation should shift in a predictable way that reflects that semantic change. This allows the model to recognize and account for shifts in user intent rather than trying to ignore them.
By moving from random perturbations to intent-preserving mechanisms—whether through model-level noise, generative modeling, or equivariant objectives—the field is shifting toward a more nuanced understanding of human behavior as a structured, semantic narrative rather than a mere collection of discrete IDs.