No Query, No Access

Wenqiang Wang, Siyuan Liang, Yangshijie Zhang, Xiaojun Jia, Hao Lin, Xiaochun CaoarXiv 2025 · DOI · arXiv

Abstract

Adversarial attacks on Natural Language Processing (NLP) models typically require either internal model access or extensive querying of APIs. This paper presents VDBA (Victim Data-based Adversarial Attack), a formidable new paradigm that requires neither. By utilizing a small set of unlabeled "victim texts" to build a hierarchical suite of substitute models, VDBA achieves high attack success rates against state-of-the-art systems like GPT-4o and Qwen2 with zero initial queries.

Introduction

Adversarial attacks—subtly modifying input text to trick a model into misclassification—pose a significant security risk. However, most existing methods are impractical for real-world deployment. White-box attacks require full access to model weights, while query-based black-box attacks are often thwarted by rate limits and detection mechanisms.

Comparison of attack paradigms: Query-based, Transfer-based, and the proposed Victim Data-based Attack (VDBA).

VDBA addresses this gap by demonstrating that an attacker only needs "victim texts"—unlabeled data similar to what the target model processes—to craft successful exploits. This "query-less" approach allows attackers to remain completely invisible until the moment of the attack.

Methodology

The VDBA framework operates through a three-stage pipeline designed to simulate the victim model's decision boundaries without direct interaction.

Overview of the VDBA framework, illustrating shadow dataset synthesis, hierarchical model design, and adversarial example generation.

1. Cluster-Guided Shadow Dataset Synthesis

In the absence of labeled data, VDBA utilizes unlabeled victim texts. These are transformed into vectors using a pre-trained embedding model (e.g., T5). K-means clustering is then applied to partition the data, assigning "pseudo-labels" to create a "shadow dataset" that approximates the target's task structure.

2. Hierarchical Substitute Model Design

To capture the nuances of a victim model's decision boundary, VDBA employs a "divide and conquer" strategy. It iteratively re-clusters data to train a hierarchy of substitute models. This ensures that even if one model is inaccurate, others in the hierarchy likely mirror the victim's logic for specific text subsets.

3. Diverse Adversarial Example Generation

VDBA employs a "committee" of established attack methods (such as BAE, Hotflip, and PSO) across its substitute models. A scoring function selects the optimal adversarial candidate that maximizes attack effectiveness while maintaining high semantic similarity to the original input.

Experimental Evaluation

The authors evaluated VDBA across several standard NLP benchmarks, including SST5 and Emotion for sentiment analysis, and AG News and TREC6 for classification tasks. To test extreme scenarios, "one-shot" sources were pulled from Go-Emotions and Rotten Tomatoes to train shadow models with minimal initial data.

Key Results

  • State-of-the-Art Performance: On the SST5 dataset, VDBA achieved an Attack Success Rate (ASR) of 52.08%, a significant improvement over traditional transfer-based methods, despite requiring zero queries.
  • LLM Vulnerability: The attack proved remarkably effective against closed-source models, achieving an ASR of 45.99% on Qwen2 and over 35% on GPT-4o.
  • Data Scarcity Resilience: Even with as few as five victim texts, VDBA maintained an ASR of over 27%, highlighting the risk posed by even minor data leaks.

Discussion and Conclusion

The core insight of VDBA is that exact label replication is unnecessary; simulating high-level decision boundaries through clustering is sufficient for successful transfer attacks. Furthermore, the attack's effectiveness with simple "one-hot" encodings suggests it does not require sophisticated hardware.

While highly effective, VDBA faces a trade-off between the number of hierarchical iterations and semantic similarity. Aggressive attacks can occasionally result in less natural text. Future research will explore extending VDBA to other modalities like image or audio and developing more robust defense mechanisms against data-based exploits.

Summary updated

Abstract

Textual adversarial attacks mislead NLP models, including Large Language Models (LLMs), by subtly modifying text. While effective, existing attacks often require knowledge of the victim model, extensive queries, or access to training data, limiting real-world feasibility. To overcome these constraints, we introduce the \textbf{Victim Data-based Adversarial Attack (VDBA)}, which operates using only victim texts. To prevent access to the victim model, we create a shadow dataset with publicly available pre-trained models and clustering methods as a foundation for developing substitute models. To address the low attack success rate (ASR) due to insufficient information feedback, we propose the hierarchical substitution model design, generating substitute models to mitigate the failure of a single substitute model at the decision boundary. Concurrently, we use diverse adversarial example generation, employing various attack methods to generate and select the adversarial example with better similarity and attack effectiveness. Experiments on the Emotion and SST5 datasets show that VDBA outperforms state-of-the-art methods, achieving an ASR improvement of 52.08\% while significantly reducing attack queries to 0. More importantly, we discover that VDBA poses a significant threat to LLMs such as Qwen2 and the GPT family, and achieves the highest ASR of 45.99% even without access to the API, confirming that advanced NLP models still face serious security risks. Our codes can be found at https://anonymous.4open.science/r/VDBA-Victim-Data-based-Adversarial-Attack-36EC/
BibTeX
@misc{wang2025no,
  author = {Wenqiang Wang and Siyuan Liang and Yangshijie Zhang and Xiaojun Jia and Hao Lin and Xiaochun Cao},
  title = {{No Query, No Access}},
  year = {2025},
  eprint = {2505.07258},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.07258},
}
Figure