Contrastive Learning for Sparse Sequential Recommendation

This direction is merged into another direction: data sparsity (data sparsity in sequential recommendation). Its last version stays readable and citable here.

Version 1 of 1192 papers203 conceptsfrozen 2026-10-04

Sequential recommendation systems aim to predict a user's next interaction based on their historical behavior. However, these systems face a fundamental hurdle: data sparsity. In a typical e-commerce or content platform, most users have only a handful of interactions, yet they choose from a catalog of millions of items. When a deep learning model like a Transformer is trained solely on these sparse "next-item" labels, it often fails to learn robust representations of user intent.

To bridge this gap, a major research direction has emerged that uses Self-Supervised Learning (SSL)—specifically Contrastive Learning—to extract more information from limited interaction data. Instead of relying only on the final click, these models learn by comparing different "views" of the same user sequence, forcing the model to identify the underlying patterns that define a user's preference.

The Contrastive Breakthrough

The core intuition behind this direction, pioneered by researchers like Fei Sun in CL4SRec, is to create a "pretext task" that doesn't require new labels. By taking a single user sequence and applying stochastic augmentations—such as cropping a portion of the history, masking certain items, or reordering interactions—the model generates two different versions of the same history.

The model is then trained to pull the representations of these two "positive" views together in the latent space while pushing them away from "negative" views (sequences from other users). This process provides a much denser supervisory signal than simple next-item prediction, allowing models to learn stable user profiles even from very short histories.

From Random Noise to Semantic Integrity

Early methods relied on random data augmentation, but this can be "noisy." For example, if you crop the only electronics purchase from a user's history, the model might lose the very "DNA" of that user's intent.

To solve this, researchers have moved toward more "aware" forms of augmentation:

  • Equivariant Learning: Peilin Zhou and colleagues argued in ECL-SR that models should be invariant to minor tweaks (like dropout) but equivariant to invasive changes (like item substitution). This allows the model to recognize when a change actually shifts the user's intent.
  • Learnable Augmentation: Rather than using fixed rules, Jiaqing Fan introduced LMA4Rec, which uses learnable Bernoulli dropout to create views that preserve semantic integrity. Similarly, Xiuyuan Qin proposed MCLRec, which uses meta-learning to optimize the augmenters themselves.
  • Model-Level Perturbation: Instead of changing the data, Zhiwei Liu suggested in SRMA that we should perturb the neural network itself—through neuron masking or layer dropping—to create contrastive views without destroying the sequence's original patterns.

Generative Filling and Diffusion

The most recent evolution in this field moves from augmenting existing data to generating missing data. If a user's history is a fragmented puzzle, these models try to fill in the missing pieces.

Haolun Wu and others introduced CaDiRec, which uses a context-aware Diffusion Model to generate realistic, semantically consistent items to "pad" sparse sequences. By denoising user history, models like PDRec by Haokai Ma provide "soft" positive signals that help the recommender understand what a user might have liked, even if they didn't click it.

Even more radically, Zhengyi Yang proposed TDM, which strategically removes data during training to teach the model how to robustly extrapolate preferences from incomplete histories.

Incorporating Global Context

While contrastive learning often focuses on a single user's sequence ("intra-sequence"), researchers like Ziyang Wang in MCLSR have expanded this to "multi-level" contrast. By pulling in collaborative signals from user-item and item-item graphs, the model learns that if two users have similar sparse histories, their representations should be closer together.

This shift from local sequence modeling to global, contrastive representation learning has allowed sequential recommenders to overcome the "sparsity wall," making them effective even for "tail" users with minimal interaction history, as explored by Shaowei Wei in $S^4$Rec.

Papers in this version

Showing 60 of 192 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_contrastive_learning_for_sparse_sequenti_8eb0da85,
  title = {Contrastive Learning for Sparse Sequential Recommendation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/contrastive-learning-for-sparse-sequential-recommendation/rdr_8eb0da853e334521bc6d3932c547dfd6},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_contrastive_learning_for_sparse_sequenti_8eb0da85_v1,
  title = {Contrastive Learning for Sparse Sequential Recommendation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/contrastive-learning-for-sparse-sequential-recommendation/rdr_8eb0da853e334521bc6d3932c547dfd6/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}