Abstract
Recently, recommender systems have achieved significant success. However, due to their openness, they remain vulnerable to malicious attacks and natural noise. This survey provides a comprehensive overview of robustness in recommender systems, categorizing it into adversarial and non-adversarial robustness. We introduce fundamental principles, classical methods, and summarize commonly used datasets and metrics. Furthermore, we propose ShillingREC, an adversarial robustness evaluation library, to facilitate fair and efficient evaluation of attack and defense methods.
Introduction: The Vulnerability of Recommendation
Most recommendation algorithms prioritize accuracy over stability. Because these systems are "open"—allowing users to create accounts and provide feedback—they are prime targets for shilling attacks. In these scenarios, attackers inject fake user profiles to "push" (promote) or "nuke" (demote) specific items. Beyond malicious intent, non-adversarial factors like data sparsity, natural noise in implicit feedback, and data imbalance also degrade performance. The community has lacked a unified way to categorize these threats and a standardized tool to evaluate how well models stand up to them.
A Taxonomy of Robustness
The field of robust recommendation can be broadly divided into two categories: adversarial and non-adversarial robustness.
1. Adversarial Robustness
This focuses on intentional attacks designed to manipulate system outputs. These are categorized by their underlying mechanisms:
- Heuristic-based: Simple strategies like "Random" or "Average" attacks.
- Optimization-based: More sophisticated attacks that use bi-level optimization to generate effective fake data.
- GAN-based: Using Generative Adversarial Networks to create "stealthy" fake profiles that mimic real user behavior.
- RL-based: Leveraging Reinforcement Learning to optimize attack sequences.
Defenses against these threats include detection mechanisms, robust training algorithms, and hybrid methods.
2. Non-Adversarial Robustness
This addresses "natural" data issues. Key challenges include handling Data Sparsity, filtering Natural Noise in implicit feedback, and addressing Data Imbalance to ensure the model doesn't over-rely on popular items.
ShillingREC: A Standardized Evaluation Library
To address the lack of unified evaluation tools, the authors introduced ShillingREC. Built on PyTorch, this modular library allows researchers to inject fake data into clean datasets using various attack models and test them against standard models like MF, LightGCN, and SGL.
ShillingREC is unique in its support for implicit feedback (clicks/views), reflecting how modern industrial recommenders actually operate. It evaluates results using specialized metrics like Prediction Shift (PS) and Targeted Hit Rate (T-HR).
Key Insights and Results
Benchmarking via ShillingREC revealed several critical findings:
- Fragility of Collaborative Filtering: CF models are inherently vulnerable because they rely on collective user behavior, which attackers can mimic to hijack the system's "intelligence."
- The Strength of Self-Supervision: Models using unsupervised contrastive learning (like SGL) are significantly more robust. The contrastive tasks act as a filter, helping the model focus on consistent global patterns rather than local "outlier" signals from attackers.
- Attack Sophistication: GAN-based and Optimization-based attacks are far more destructive and harder to detect than traditional heuristic methods, as they generate profiles that look statistically identical to real users.
- Scale Matters: Smaller or sparser datasets are much easier to manipulate, as fake signals can more easily drown out real user data.
Future Directions
Despite progress, several challenges remain. Most research relies on simulated fake users; there is a dire need for benchmarks based on actual, real-world fraudulent behavior. Additionally, robust algorithms must be optimized for high-timeliness production environments where efficiency is critical. Finally, developing a "universal" defense strategy that works across different attack types remains a primary goal for the community.
The ShillingREC project is available at: https://github.com/chengleileilei/ShillingREC
Summary updated