Mitigating Preference Leakage via Strict Estimator Separation for Normative Generative Ranking

Dalia Nahhas, Xiaohao Cai, Imran Razzak, Shoaib Jameel

In the rapidly evolving world of Generative Information Retrieval (GenIR), we’ve moved past the "can it generate?" phase and into the "which one is best?" phase. When an AI produces ten different versions of a story, selecting the one that best fits specific cultural or moral norms is a high-stakes ranking problem. However, a new paper by Nahhas et al. reveals a systemic flaw in how we evaluate these systems: preference leakage.

TL;DR

Researchers have identified a "circularity" crisis in AI evaluation where using the same model family for both training and judging leads to inflated, unreliable performance. By introducing a "strictly orthogonal" two-judge framework and a new benchmark (NGR-33k), they demonstrate that subtle cultural nuances can be distilled into efficient rankers that actually outperform their "teachers" by filtering out evaluator bias.

What Problem Does This Paper Address?

Current GenIR systems often use "LLM-as-a-Judge" to evaluate outputs. The problem is preference leakage: if you use a Llama-based model to provide training signals and then use another Llama-based model to evaluate the final product, the high scores might just reflect the model family’s internal "self-preference" rather than actual quality.

This is particularly problematic for normative criteria like cultural relevance. Unlike topical relevance (e.g., "is this story about a cat?"), cultural relevance is "thick" and subtle. A standard ranker might see the word "hummus" and think a story is culturally Arab, whereas a human (or a sophisticated judge) would look for deeper enactments, such as specific rituals of hospitality or refusal. Without strict separation between the "Teacher" and the "Evaluator," we risk building systems that are merely good at mimicking a specific judge's quirks rather than grounding themselves in real-world norms.

Methods

The authors propose a leakage-free two-judge framework designed to break the circularity of AI evaluation:

  1. Strict Estimator Separation: They use two entirely different model families for different roles. Judge B (LLaMA-3.1-8B) acts as the Supervisor, providing noisy training labels. Judge A (Yi-1.5-9B) acts as the independent Evaluator. No information—no gradients, no prompts, no scores—flows from Judge A to the training loop.
  2. Generate-then-Rank: They used Gemma-2-9B to generate a massive pool of stories based on a "triad" of constraints: Target Age, Moral Theme, and Cultural Context.
  3. Cross-Encoder to Bi-Encoder Distillation: They trained a high-capacity Cross-Encoder (Teacher) on Judge B’s scores. They then distilled that knowledge into a BGE-M3 Bi-Encoder (Student).
  4. Intersection Filtering: To ensure a fair fight, they only evaluated stories where both judges successfully provided a valid score, creating a "unified intersection pool."

Datasets

  • NGR-33k (Normative Generative Ranking): A new benchmark introduced in this paper containing 33,052 culturally grounded children's stories. It covers 7 age groups, 7 moral themes (e.g., bravery, honesty), and 7 cultural contexts (e.g., Arab, South Asian, African).
  • Moral Stories: A human-curated dataset used to validate that the LLM judges actually align with human social norms.
  • SSGEN: Used as an "out-of-distribution" control to ensure the judges weren't just agreeing by accident on random text.

Key Insights and Ideas

  • The Regularization Hypothesis: In a surprising twist, the Student (Bi-Encoder) significantly outperformed its Teacher (Cross-Encoder) under independent evaluation (nDCG@5 of 0.771 vs 0.577). The authors posit that the Bi-Encoder’s simpler architecture acts as a "bottleneck" that filters out the Teacher's tendency to overfit to the specific idiosyncrasies and "noise" of Judge B.
  • Normative Feature Suppression: The study found that standard "off-the-shelf" embedding models (like vanilla BGE-M3) struggle with cultural nuances because topical signals (keywords) overshadow latent normative signals. Direct fine-tuning is often unstable, making distillation from a Cross-Encoder a necessary "denoising" step.
  • Lexical Failure: Traditional search metrics like BM25 are almost useless for this task. They tend to reward "surface-level stereotypes" (mentioning specific foods) rather than the "deep normative grounding" (how characters actually behave) that the LLM judges look for.

Key Results

  • Distillation Success: The distilled BGE-M3 model achieved a 0.771 nDCG@5, a massive leap over the random baseline (0.571) and the un-distilled version.
  • Teacher Overfitting: The Cross-Encoder, despite being more powerful, fell to 0.577 nDCG@5 under Judge A’s evaluation. This proves it had "leaked" Judge B’s specific preferences which didn't generalize to the independent evaluator.
  • Human Alignment: On the human-curated Moral Stories dataset, Judge A and Judge B showed strong agreement (Spearman $\rho$ = 0.653), suggesting the framework captures genuine normative values rather than just "synthetic artifacts."

Limitations and Future Work

The authors acknowledge that while they separated the models, both LLaMA and Yi likely share massive amounts of pre-training data (like CommonCrawl). This means they might still share "latent societal priors" that aren't fully independent.

Future work could explore:

  • Using even more diverse model architectures (e.g., non-Transformer models) to further reduce family bias.
  • Expanding the "triad" of constraints to include more complex social dimensions beyond age, morals, and culture.
  • Investigating why the "bottleneck" effect of Bi-Encoders is so effective at noise filtering in other normative ranking tasks.

Summary updated

Abstract

In Generative Information Retrieval (GenIR), the bottleneck has shifted from generation to the selection of candidates, particularly for normative criteria such as cultural relevance. Current LLM-as-a-Judge evaluations often suffer from circularity and preference leakage, where overlapping supervision and evaluation models inflate performance. We address this by formalising cultural relevance as a within-query ranking task and introducing a leakage-free two-judge framework that strictly separates supervision (Judge B) from evaluation (Judge A). On a new benchmark of 33,052 (NGR-33k) culturally grounded stories, we find that while classical baselines yield only modest gains, a dense bi-encoder distilled from a Judge-B-supervised Cross-Encoder is highly effective. Although the Cross-Encoder provides a strong supervision signal for distillation, the distilled BGE-M3 model substantially outperforms it under leakage-free Judge~A evaluation. We validate our framework on the human-curated Moral Stories dataset, showing strong alignment with human norms. Our results demonstrate that rigorous evaluator separation is a prerequisite for credible GenIR evaluation, proving that subtle cultural preferences can be distilled into efficient rankers without leakage.
BibTeX
@misc{nahhas2026mitigating,
  author = {Dalia Nahhas and Xiaohao Cai and Imran Razzak and Shoaib Jameel},
  title = {{Mitigating Preference Leakage via Strict Estimator Separation for Normative Generative Ranking}},
  year = {2026},
  eprint = {2602.20800},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.20800},
}
Figure