Abstract
Large Language Models (LLMs) have demonstrated strong performance in information retrieval tasks like passage ranking. Our research examines how instruction-following capabilities in LLMs interact with multi-document comparison tasks, identifying what we term the "Ranking Blind Spot", a characteristic of LLM decision processes during comparative evaluation. We analyze how this ranking blind spot affects LLM evaluation systems through two approaches: Decision Objective Hijacking, which alters the evaluation goal in pairwise ranking systems, and Decision Criteria Hijacking, which modifies relevance standards across ranking schemes. These approaches demonstrate how content providers could potentially influence LLM-based ranking systems to affect document positioning. These attacks aim to force the LLM ranker to prefer a specific passage and rank it at the top. Malicious content providers can exploit this weakness, which helps them gain additional exposure by attacking the ranker. In our experiment, We empirically show that the proposed attacks are effective in various LLMs and can be generalized to multiple ranking schemes. We also found stronger LLMs are more vulnerable to these attacks. Our code is available at: https://github.com/blindspotorg/RankingBlindSpot
Introduction
As Large Language Models (LLMs) increasingly serve as the backbone for modern search engines and recommendation systems, they are tasked with ranking documents based on relevance. Unlike traditional rankers that rely on numerical scoring, LLM-based rankers often perform "comparative evaluation," analyzing multiple documents simultaneously to determine their relative order.
However, this process reveals a fundamental vulnerability: the Ranking Blind Spot. This vulnerability arises because LLMs struggle to distinguish between "system instructions" (e.g., "Rank these documents by relevance") and the "data" (the text within the documents). Malicious content providers can exploit this by embedding "prompt injections" that hijack the LLM’s decision-making logic, forcing it to ignore its original ranking criteria in favor of the attacker's goals.
Methodology
The research introduces a framework called Decision Hijacking, which consists of two primary attack strategies designed to exploit the Ranking Blind Spot:
- Decision Objective Hijacking (DOH): A "task substitution" attack that changes the model's fundamental objective. Instead of evaluating relevance, the model is tricked into performing a simpler task, such as ranking a document first if it contains a specific marker.
- Decision Criteria Hijacking (DCH): A "concept redefinition" attack that alters how the model judges documents. It redefines "relevance" itself, instructing the model that the presence of a specific marker is the highest priority for determining value.
These attacks rely on two key LLM weaknesses: Boundary Confusion (inability to separate document content from instructions) and Priority Confusion (difficulty prioritizing the system prompt over injected instructions). The framework was tested across three common ranking paradigms: Pairwise, Listwise, and Setwise.
Experimental Setup
The study utilized industry-standard benchmarks and real-world data:
- TREC-DL 2019 & 2020: Standard benchmarks for evaluating large-scale information retrieval systems.
- Google Trends: To ensure realistic validation, the authors used trending queries (e.g., "iPhone 16", "Paris Olympics") and retrieved top web pages via Google to test attacks in a live-web context.
- Models Evaluated: A diverse range of LLMs including GPT-4.1-mini, Llama-3.3-70B, Qwen3, and Gemma-3.
Results and Discussion
The quantitative results demonstrate a near-total collapse of ranking integrity under attack. For top-tier models like GPT-4.1-mini and Llama-3.3-70B, the attack success rate—moving a low-relevance document to the #1 spot—frequently exceeded 99%.
The Capability Paradox
A striking finding of this research is the "Capability Paradox." While larger, more capable models are typically more robust, the opposite is true here: stronger models like GPT-4 and Llama-3-70B are more susceptible to hijacking. Their superior instruction-following capabilities make them more likely to obey malicious instructions embedded within document content.
Impact on Ranking Quality
Using the NDCG@10 metric, the attacks caused catastrophic performance degradation. For instance, Llama-3-70B’s score on the DL19 dataset plummeted from 74.30 to 7.38. The vulnerability was found to be:
- Scheme Agnostic: Effective across Pairwise, Listwise, and Setwise ranking.
- Position Agnostic: Effective regardless of where the malicious prompt is placed in a document.
Conclusion
The "Ranking Blind Spot" represents a significant security challenge for LLM-based information retrieval. Current models lack a strict instructional hierarchy, treating all text in the context window with similar authority. This allows document content to override system-level programming.
The authors conclude that simple prompt "patching" is insufficient. Future work must focus on architectural changes, including instructional separation (using privileged channels for system prompts), adversarial fine-tuning, and semantic anomaly detection to identify and mitigate manipulative intent within document content.
Summary updated