LLM-EDT: Large Language Model Enhanced Cross-domain Sequential Recommendation with Dual-phase Training

Ziwei Liu, Qidong Liu, Wanyu Wang, Yejing Wang, Tong Xu, Wei Huang, Chong Chen, Peng Chuan, Xiangyu Zhao

Abstract

Cross-domain Sequential Recommendation (CDSR) has been proposed to enrich user-item interactions by incorporating information from various domains. Despite current progress, the imbalance issue and transition issue hinder further development of CDSR. The former one presents a phenomenon that the interactions in one domain dominate the entire behavior, leading to difficulty in capturing the domain-specific features in the other domain. The latter points to the difficulty in capturing users' cross-domain preferences within the mixed interaction sequence, resulting in poor next-item prediction performance for specific domains. With world knowledge and powerful reasoning ability, Large Language Models (LLMs) partially alleviate the above issues by performing as a generator and an encoder. However, current LLMs-enhanced CDSR methods are still under exploration, which fail to recognize the irrelevant noise and rough profiling problems. Thus, to make peace with the aforementioned challenges, we proposed an LLMs Enhanced Cross-domain Sequential Recommendation with Dual-phase Training (LLM-EDT). To address the imbalance issue while introducing less irrelevant noise, we first propose the transferable item augmenter to adaptively generate possible cross-domain behaviors for users. Then, to alleviate the transition issue, we introduce a dual-phase training strategy to empower the domain-specific thread with a domain-shared background. As for the rough profiling problem, we devise a domain-aware profiling module to summarize the user's preference in each domain and adaptively aggregate them to generate comprehensive user profiles. The experiments on three public datasets validate the effectiveness of our proposed LLM-EDT.

Introduction

Sequential Recommender Systems (SRS) aim to predict a user's next interaction based on historical behavior. However, these systems often face significant challenges in cross-domain scenarios. Two primary hurdles are the Imbalance Issue—where data sparsity in one domain makes it difficult to capture specific features—and the Transition Issue, which involves the difficulty of tracking preference shifts as users move between disparate categories (e.g., from "Clothing" to "Electronics").

While Large Language Models (LLMs) offer potential solutions through their reasoning capabilities, existing LLM-enhanced methods often struggle with Irrelevant Noise (hallucinations) and Rough Profiling (oversimplified summaries). LLM-EDT is designed to address these limitations by leveraging LLMs as "teachers" to generate high-quality data and profiles offline, avoiding real-time latency.

The LLM-EDT Framework

The proposed LLM-EDT framework consists of three core components designed to bridge data gaps and refine user understanding.

The complete LLM-EDT framework architecture, illustrating the Transferable Item Augmenter, Two-Stage Training, and Domain-aware Profiling modules.

1. Transferable Item Augmenter

To combat the imbalance issue without introducing noise, LLM-EDT employs a "Sample-Filter-Insert" pipeline. Instead of random augmentation, it generates potential items for sparse domains and uses LLM-based semantic embeddings to filter out irrelevant suggestions. These items are then strategically inserted into the user's sequence as "bridges" between domains, specifically following related items in the other domain.

2. Dual-phase Training Framework

The training process is split into two distinct stages:

  • Global Pre-training: The model (based on SASRec) is trained on mixed, augmented sequences to establish a broad, domain-shared background.
  • Domain Fine-tuning: The model is specialized on domain-specific threads, allowing it to retain global context while mastering the nuances of individual categories.

3. Domain-aware Profiling

To move beyond rough profiling, LLM-EDT uses a Summarize-Reform-Analyze pipeline. User behaviors are clustered via K-means and summarized. The LLM then reconciles these summaries based on the actual interaction ratios between domains, producing a comprehensive and nuanced user profile.

Experimental Evaluation

The framework was evaluated on three Amazon dataset pairs: Cloth-Sport, Electronic-Cell Phone, and Food-Kitchen. These datasets are known for the imbalance issue where one category often dwarfs the other in interaction volume.

Distribution of user counts across different domain ratios, comparing raw data to augmented data to demonstrate how LLM-EDT addresses the imbalance issue.

As shown in Figure 1, the augmentation strategy significantly shifts the user distribution, providing more data for users who previously had very sparse interactions in one domain. Key performance highlights include:

  • Significant Gains: LLM-EDT achieved improvements in Hit Rate (H@10) and NDCG (N@10) ranging from 2.81% to 26.96% over state-of-the-art baselines.
  • Cold-Start Performance: The model showed marked improvements for users with very short sequences, validating the effectiveness of the item augmenter.
  • Efficiency: By caching LLM outputs, the system maintains the inference speed of a traditional lightweight model (SASRec), with only a marginal increase in parameter count (4.12M vs 3.99M).

Conclusion

LLM-EDT demonstrates that LLMs can significantly enhance cross-domain recommendations by acting as sophisticated data generators and profilers during the training phase. This approach bypasses the latency issues typically associated with LLMs in production. Future work will look into extending this framework to handle more than two domains and further automating the profiling instructions to capture even more diverse user behaviors.

Summary updated

Abstract

Cross-domain Sequential Recommendation (CDSR) has been proposed to enrich user-item interactions by incorporating information from various domains. Despite current progress, the imbalance issue and transition issue hinder further development of CDSR. The former one presents a phenomenon that the interactions in one domain dominate the entire behavior, leading to difficulty in capturing the domain-specific features in the other domain. The latter points to the difficulty in capturing users' cross-domain preferences within the mixed interaction sequence, resulting in poor next-item prediction performance for specific domains. With world knowledge and powerful reasoning ability, Large Language Models (LLMs) partially alleviate the above issues by performing as a generator and an encoder. However, current LLMs-enhanced CDSR methods are still under exploration, which fail to recognize the irrelevant noise and rough profiling problems. Thus, to make peace with the aforementioned challenges, we proposed an LLMs Enhanced Cross-domain Sequential Recommendation with Dual-phase Training ({LLM-EDT}). To address the imbalance issue while introducing less irrelevant noise, we first propose the transferable item augmenter to adaptively generate possible cross-domain behaviors for users. Then, to alleviate the transition issue, we introduce a dual-phase training strategy to empower the domain-specific thread with a domain-shared background. As for the rough profiling problem, we devise a domain-aware profiling module to summarize the user's preference in each domain and adaptively aggregate them to generate comprehensive user profiles. The experiments on three public datasets validate the effectiveness of our proposed LLM-EDT. To ease reproducibility, we have released the detailed code online at {https://anonymous.4open.science/r/LLM-EDT-583F}.
BibTeX
@misc{liu2025llm,
  author = {Ziwei Liu and Qidong Liu and Wanyu Wang and Yejing Wang and Tong Xu and Wei Huang and Chong Chen and Peng Chuan and Xiangyu Zhao},
  title = {{LLM-EDT: Large Language Model Enhanced Cross-domain Sequential Recommendation with Dual-phase Training}},
  year = {2025},
  eprint = {2511.19931},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.19931},
}
Figure