Attacking Pre-trained Recommendation

Yiqing Wu, Ruobing Xie, Zhao Zhang, Yongchun Zhu, Fuzhen Zhuang, Jie Zhou, Yongjun Xu, Qing He

Abstract

As recommendation systems transition toward a "pre-train then fine-tune" paradigm, new security vulnerabilities emerge in the AI supply chain. This paper investigates the threat posed by malicious model providers who embed backdoors into pre-trained recommendation models. We demonstrate that these backdoors can significantly manipulate item exposure for specific user groups while remaining invisible to standard monitoring. Our findings highlight a critical need for integrity verification in shared AI models.

Introduction

Traditional recommendation system attacks typically involve "shilling" or "poisoning" where malicious users inject fake interaction logs to boost specific products. However, the shift toward Pre-trained Recommendation (PR) introduces a more potent threat actor: the Model Provider. Many downstream platforms lack the massive datasets required to train state-of-the-art models from scratch, opting instead to fine-tune or prompt-tune models provided by third parties.

This research explores a scenario where a provider intentionally compromises a model during pre-training. By hiding a backdoor trigger, the attacker can secretly promote target items—such as specific brands or propaganda—without affecting the model's performance on general tasks. This creates a "supply chain" vulnerability where the malicious intent is baked into the model weights before they ever reach the platform.

The supply chain attack pipeline: a malicious model provider delivers a backdoored model to downstream platforms.

Attack Methodologies

The authors propose two primary strategies to exploit the PR paradigm:

1. Basic Replacement Attack (BRA)

BRA is a data-poisoning method applied during pre-training. The attacker generates malicious user sequences by replacing legitimate items with "target items" from the same category. This semantic similarity ensures the sequences appear natural, bypassing standard data-cleaning filters while establishing a strong association between specific contexts and the target item.

2. Prompt-enhanced Attack (PEA)

PEA is a more advanced "meta-attack" designed to survive prompt-tuning, a common efficiency-focused adaptation method. The attacker simulates the downstream tuning process on their own data to generate "fake prompts." By freezing these prompts and tuning the model to respond to them via BRA, the attacker "teaches" the model to associate specific prompt patterns with the promotion of target items. This allows the attack to persist even when the platform uses its own private data for tuning.

Experimental Setup

The study utilizes two large-scale Alibaba e-commerce datasets: CIKM (130k items) and AliEC (109k items). To ensure a realistic transfer-learning scenario, the data is partitioned such that the Model Provider and the downstream Platforms have zero overlapping users, mimicking a true cross-platform deployment.

Key Results and Insights

The effectiveness of these attacks is measured through Hit Rate (H@K), Normalized Discounted Cumulative Gain (N@K), and a novel Invisibility ($\lambda$) metric. Invisibility quantifies the gap in attack success between target users and non-target users; a high $\lambda$ indicates a surgical attack that is difficult to detect through general system monitoring.

Performance comparison of clean and attacked models across CIKM and AliEC datasets, highlighting massive exposure spikes and high invisibility scores.

Massive Exposure Spikes

The results show that the BRA method can increase the Hit Rate of target items by several hundred times compared to a clean baseline. For instance, on the CIKM dataset, the H@5 for target users jumps from 0.002 (Clean) to 0.755 (BRA). Even after fine-tuning (BRA$^{FT}$), the exposure remains significantly higher than the baseline.

Surgical Precision and Invisibility

The PEA method demonstrates remarkable surgical precision. On the AliEC dataset, PEA achieved an invisibility score of 51.2x, meaning the attack was over 50 times more effective on the intended demographic than on the general population. This high degree of specificity makes the backdoor nearly impossible to identify through aggregate performance metrics.

Vulnerability of Prompt-Tuning

The study finds that prompt-tuning actually exacerbates vulnerability. Because prompt-tuning keeps the underlying pre-trained weights static, it preserves the malicious triggers embedded during pre-training more effectively than full fine-tuning. This suggests that the most efficient ways to adapt models are also the most dangerous from a security perspective.

Conclusion

This research serves as a critical warning for the AI industry. As the "pre-train then fine-tune" workflow becomes standard, the integrity of pre-trained weights becomes as important as their accuracy. Current detection methods struggle to identify sophisticated attacks like PEA, which often yield F1 scores near zero in detection tests. Future work must focus on developing robust verification and defense mechanisms to secure the AI supply chain against malicious providers.

Summary updated

Abstract

Recently, a series of pioneer studies have shown the potency of pre-trained models in sequential recommendation, illuminating the path of building an omniscient unified pre-trained recommendation model for different downstream recommendation tasks. Despite these advancements, the vulnerabilities of classical recommender systems also exist in pre-trained recommendation in a new form, while the security of pre-trained recommendation model is still unexplored, which may threaten its widely practical applications. In this study, we propose a novel framework for backdoor attacking in pre-trained recommendation. We demonstrate the provider of the pre-trained model can easily insert a backdoor in pre-training, thereby increasing the exposure rates of target items to target user groups. Specifically, we design two novel and effective backdoor attacks: basic replacement and prompt-enhanced, under various recommendation pre-training usage scenarios. Experimental results on real-world datasets show that our proposed attack strategies significantly improve the exposure rates of target items to target users by hundreds of times in comparison to the clean model. The source codes are released in https://github.com/wyqing20/APRec.
BibTeX
@inproceedings{wu2023attacking,
  author = {Yiqing Wu and Ruobing Xie and Zhao Zhang and Yongchun Zhu and Fuzhen Zhuang and Jie Zhou and Yongjun Xu and Qing He},
  title = {{Attacking Pre-trained Recommendation}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2023},
  url = {https://openreview.net/forum?id=IDMKDVq0_C},
}
Figure