Abstract
Information Retrieval (IR) systems often struggle with messy implicit feedback, characterized by extreme class imbalance and data sparsity. This paper introduces AdvIR, a framework that leverages adversarial perturbations and selective sampling to enhance semi-supervised IR. By generating "hard" training examples and identifying informative non-clicked documents, AdvIR achieves significant performance gains and extreme data efficiency, outperforming baselines like IRGAN with only a fraction of the labeled data.
Introduction
Modern IR systems, such as web search engines and recommendation platforms, rely heavily on implicit user feedback (e.g., clicks). However, this data presents three primary challenges:
- Imbalance: Non-clicked documents vastly outnumber clicked ones, yet most are irrelevant and offer little learning signal.
- Sparsity: Positive labels (clicks) are rare, hindering model generalization.
- Model Vulnerability: Neural IR models are often sensitive to small input perturbations, making them vulnerable to ranking errors.
The AdvIR framework addresses these issues by using adversarial techniques to strengthen positive signals and identify the most informative negative examples.
Methodology
The AdvIR framework consists of several key components designed to improve robustness and efficiency:
Adversarial Training
The model generates adversarial perturbations—calculated noise added to input features—to maximize training loss. By learning to remain accurate under these "attacks," the model develops a smoother and more robust decision boundary.
Adversarial Sampling
Instead of random negative sampling, AdvIR uses a softmax-based approach to identify "difficult" negatives—documents the model incorrectly predicts as highly relevant. This ensures the model learns from the most informative examples.
Selective Virtual Adversarial Training (AdvIR-SVAT)
To reduce the computational cost of standard Virtual Adversarial Training (VAT), the authors propose Selective VAT. This method applies smoothing only to the most informative examples identified through adversarial sampling, maintaining performance while improving efficiency.
Key Results
AdvIR demonstrates superior performance across web search, item recommendation, and question-answering tasks.
As shown in Figure 1, AdvIR with Adversarial Sampling (AS) consistently outperforms the IRGAN baseline and AdvIR with Uniform Sampling (US). The gains are most notable in high-rank precision; for web search, AdvIR achieved a 21.3% relative improvement in Precision@3 over the ReLU-based IRGAN.
The framework's data efficiency is a standout feature. As illustrated in Figure 4, AdvIR achieves higher NDCG@5 scores using only 5% of labeled data than IRGAN does with the full dataset. Furthermore, the learning curves (panels a and b) indicate that AdvIR converges to high precision significantly faster than its competitors, often reaching stability within fewer training epochs.
Conclusion
AdvIR provides a robust solution for semi-supervised information retrieval by combining adversarial training with intelligent sampling. Its ability to achieve state-of-the-art results with minimal labeled data makes it a powerful tool for real-world IR applications where high-quality labels are scarce. Future work may explore the interpretability of adversarial perturbations and address cold-start issues in recommendation systems.
Summary updated