RankSteer: Activation Steering for Pointwise LLM Ranking

Yumeng Wang, Catherine Chen, Suzan VerbernearXiv 2026 · arXiv · OpenReview

TL;DR

This paper introduces RankSteer, a novel method for improving the performance of Large Language Models (LLMs) in zero-shot ranking tasks. By directly manipulating internal model activations, RankSteer can steer LLMs towards better relevance judgments without requiring prompt engineering or model retraining, demonstrating that significant ranking potential remains untapped in current LLMs.

What Problem Does This Paper Address?

Large Language Models (LLMs) are increasingly used for information retrieval tasks like ranking documents based on relevance to a query. However, their performance in these "zero-shot" scenarios (where they haven't been explicitly trained for the specific ranking task) is highly sensitive to how they are prompted. Specifically, the way an LLM is instructed to "act" (e.g., as a "reliable" vs. "careless" assistant) can drastically change its ranking quality. Previous research suggests these "role-play" instructions are processed separately from the actual query and document content within the LLM's internal workings. This raises a key question: can we directly influence the LLM's ranking decisions by manipulating these internal signals, rather than relying on fragile prompt engineering? The authors aim to answer this by developing a method to steer LLM activations for improved pointwise ranking.

Methods

RankSteer is a post-hoc activation steering framework designed for zero-shot pointwise LLM ranking. The core idea is to identify and manipulate distinct "directions" in the LLM's internal representation space that correspond to different aspects of the ranking process. They define three key directions:

  1. Decision Direction: This direction captures how the LLM's internal states are mapped to a final "Yes/No" relevance judgment. It's derived from the model's output weights for these tokens and is input-agnostic, representing a global bias towards making a decision.
  2. Evidence Direction: This direction captures the actual relevance signals from the query and document that are present in the LLM's hidden states but not fully utilized by the decision-making part. It's extracted using contrastive examples of relevant and irrelevant documents that the LLM already ranks correctly, and then orthogonalized to remove the influence of the decision direction.
  3. Role Direction: This direction captures the influence of role-play instructions on the LLM's behavior. It's extracted using contrastive prompts with different role instructions but the same query-document pair, and is orthogonalized against both the decision and evidence directions to ensure it acts as a behavioral modulator rather than injecting new relevance information.

At inference time, RankSteer applies interventions by projecting the LLM's hidden states onto these directions and adjusting them. This allows for fine-grained control over the model's ranking behavior without altering its weights. The steering is applied layer-wise, allowing for nuanced control throughout the model's processing.

Datasets

The authors evaluate RankSteer on two main benchmark collections:

  • TREC Deep Learning (DL) [5]: Used for both constructing steering vectors and for evaluation. They specifically mention TREC DL 2019 and TREC DL 2020.
  • BEIR [32]: A collection of eight diverse datasets (Covid, Touche, Signal, News, SciFact, Robust04, DBPedia, and NFCorpus) used for evaluating out-of-distribution generalization.

Anchor queries for constructing steering vectors were primarily drawn from TREC DL 2019.

Key Insights and Ideas

  • Disentangled Ranking Signals: LLMs appear to encode distinct internal "directions" for making relevance decisions, processing evidence, and responding to role-play instructions. These signals can be separated and manipulated.
  • Activation Steering as a Powerful Tool: Directly manipulating LLM activations offers a more precise and effective way to control ranking behavior than traditional prompt engineering, especially in low-resource settings.
  • Under-utilized Ranking Capacity: Even with simple zero-shot prompts, LLMs possess substantial latent ranking capabilities that can be unlocked through activation steering.
  • Geometric Interpretation of Steering: Steering doesn't fundamentally change the LLM's ranking geometry but rather stabilizes it, reducing dispersion and making the ranking process more coherent and predictable.
  • Sample Efficiency: RankSteer achieves significant performance gains using a very small number of anchor queries (as few as 5), making it highly sample-efficient compared to methods like fine-tuning.

Key Results

  • RankSteer consistently improves ranking quality (measured by nDCG@10) across various LLMs (Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3) and datasets, often outperforming other state-of-the-art pointwise ranking methods.
  • On Llama-3.1-8B-Instruct, RankSteer achieved performance comparable to or even surpassing pairwise, setwise, and listwise methods on several datasets, despite operating strictly within the pointwise paradigm.
  • For example, on TREC DL 2020, RankSteer improved the nDCG@10 of Mistral-7B-Instruct-v0.3 from 0.5082 to 0.6355 using only 5 anchor queries.
  • An ablation study showed that combining all three steering directions (decision, evidence, and role) yielded the best overall performance, highlighting their complementary effects.
  • Compared to fine-tuning with the same anchor data, RankSteer achieved larger gains in nDCG@10 with significantly less effort (no weight updates).

Limitations and Future Work

  • Sensitivity to Anchor Set: While RankSteer is only moderately sensitive to the specific anchor query set, the magnitude of improvement can vary. The authors suggest a multi-fold selection procedure to mitigate this.
  • Binary Accuracy Trade-off: Steering can sometimes decrease binary accuracy, as it rescales decision scores and might shift the optimal threshold for a "Yes/No" judgment. This suggests a trade-off between overall ranking quality and precise binary classification.
  • Future Work: The authors suggest extending activation steering methods to other ranking paradigms and retrieval settings. They also highlight the potential for developing more sophisticated steering strategies.

Summary updated

Abstract

Large language models (LLMs) have recently shown strong performance as zero-shot rankers, yet their effectiveness is highly sensitive to prompt formulation, particularly role-play instructions. Prior analyses suggest that role-related signals are encoded along activation channels that are largely separate from query-document representations, raising the possibility of steering ranking behavior directly at the activation level rather than through brittle prompt engineering. In this work, we propose RankSteer, a post-hoc activation steering framework for zero-shot pointwise LLM ranking. We characterize ranking behavior through three disentangled and steerable directions in representation space: a \textbf{decision direction} that maps hidden states to relevance scores, an \textbf{evidence direction} that captures relevance signals not directly exploited by the decision head, and a \textbf{role direction} that modulates model behavior without injecting relevance information. Using projection-based interventions at inference time, RankSteer jointly controls these directions to calibrate ranking behavior without modifying model weights or introducing explicit cross-document comparisons. Experiments on TREC DL 20 and multiple BEIR benchmarks show that RankSteer consistently improves ranking quality using only a small number of anchor queries, demonstrating that substantial ranking capacity remains under-utilized in pointwise LLM rankers. We further provide a geometric analysis revealing that steering improves ranking by stabilizing ranking geometry and reducing dispersion, offering new insight into how LLMs internally represent and calibrate relevance judgments.
BibTeX
@misc{wang2026ranksteer,
  author = {Yumeng Wang and Catherine Chen and Suzan Verberne},
  title = {{RankSteer: Activation Steering for Pointwise LLM Ranking}},
  year = {2026},
  eprint = {2602.03422},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.03422},
}
Figure