No article text was recorded for this version.
LLM-as-a-judge bias
Version 1 of 1704 papers723 conceptsfrozen 2026-10-04
Papers in this version
- Blog: GRainsaCK: a Comprehensive Software Library for Benchmarking Explanations of Link Prediction Tasks on Knowledge Graphs
- Blog: A Literature Review and Framework for Human Evaluation of Generative Large Language Models in Healthcare
- Blog: FACTS About Building Retrieval Augmented Generation-based Chatbots
- Blog: RPGBENCH: Evaluating Large Language Models as Role-Playing Game Engines
- Blog: Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
- Blog: When Scanners Lie: Evaluator Instability in LLM Red-Teaming
- Blog: When English Rewrites Local Knowledge: Global Narrative Dominance in Large Language Models
- Blog: On Learning to Summarize with Large Language Models as References
- Blog: FB-Bench: A Fine-Grained Multi-Task Benchmark for Evaluating LLMs' Responsiveness to Human Feedback
- Blog: LLM-Eval: Unified Multi-Dimensional Automatic Evaluation for Open-Domain Conversations with Large Language Models
- Blog: Q-Ponder: A Unified Training Pipeline for Reasoning-based Visual Quality Assessment
- Blog: AgentHER: Hindsight Experience Replay for LLM Agent Trajectory Relabeling
- Blog: Optimizing the role of human evaluation in LLM-based spoken document summarization systems
- Blog: Towards Provably Unbiased LLM Judges via Bias-Bounded Evaluation
- Blog: U-MATH: A University-Level Benchmark for Evaluating Mathematical Skills in Large Language Models
- Blog: Benchmarking Cognitive Biases in Large Language Models as Evaluators
- Blog: The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge
- Blog: Xwin-LM: Strong and Scalable Alignment Practice for LLMs
- Blog: AURORA:Automated Training Framework of Universal Process Reward Models via Ensemble Prompting and Reverse Verification
- Blog: Conversational Education at Scale: A Multi-LLM Agent Workflow for Procedural Learning and Pedagogic Quality Assessment
- Blog: A Survey on Evaluation of Multimodal Large Language Models
- Blog: Structure-Conditional Minimum Bayes Risk Decoding
- Blog: This prompt is measuring \textlessmask\textgreater: evaluating bias evaluation in language models
- Blog: Beyond One-Preference-for-All: Multi-Objective Direct Preference Optimization for Language Models
- Blog: Sparks of Science: Hypothesis Generation Using Structured Paper Data
- Blog: MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Constructing Moral Discussions
- Blog: MermaidSeqBench: An Evaluation Benchmark for LLM-to-Mermaid Sequence Diagram Generation
- Blog: CodeArena: Evaluating and Aligning CodeLLMs on Human Preference
- Blog: Search Augmented Instruction Learning
- Blog: SpecEval: Evaluating Model Adherence to Behavior Specifications
- Blog: S-Eval: Automatic and Adaptive Test Generation for Benchmarking Safety Evaluation of Large Language Models
- Blog: ReFIne: A Framework for Trustworthy Large Reasoning Models with Reliability, Faithfulness, and Interpretability
- Blog: xVerify: Efficient Answer Verifier for Reasoning Model Evaluations
- Blog: AirQA: A Comprehensive QA Dataset for AI Research with Instance-Level Evaluation
- Blog: CultureCLIP: Empowering CLIP with Cultural Awareness through Synthetic Images and Contextualized Captions
- Blog: Align Beyond Prompts: Evaluating World Knowledge Alignment in Text-to-Image Generation
- Blog: Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing
- Blog: DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services
- Blog: Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints
- Blog: SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery
- Blog: ECCO: Evidence-Driven Causal Reasoning for Compiler Optimization
- Blog: From Facts to Conclusions : Integrating Deductive Reasoning in Retrieval-Augmented LLMs
- Blog: Meronymic Ontology Extraction via Large Language Models
- Blog: Single- vs. Dual-Prompt Dialogue Generation with LLMs for Job Interviews in Human Resources
- Blog: CAMEL: Communicative Agents for "Mind" Exploration of Large Scale Language Model Society
- Blog: OAEI-LLM-T: A TBox Benchmark Dataset for Understanding Large Language Model Hallucinations in Ontology Matching
- Blog: A Chinese Prompt Attack Dataset for LLMs with Evil Content
- Blog: ULTRABENCH: Benchmarking LLMs under Extreme Fine-grained Text Generation
- Blog: Know Your Place: Diagnosing Implicit Social Adaptation Failures in Chinese Large Language Models
- Blog: M3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning
- Blog: Towards Reasoning Ability of Small Language Models
- Blog: Beyond Ideal Instruction: A Comprehensive Framework for Evaluating LLMs in Realistic Interactions
- Blog: When Instructions Multiply: Measuring and Estimating LLM Capabilities of Multiple Instructions Following
- Blog: Multimodal Safety Is Asymmetric: Cross-Modal Exploits Unlock Black-Box MLLMs Jailbreaks
- Blog: M-IFEval: Multilingual Instruction-Following Evaluation
- Blog: Online Rubrics Elicitation from Pairwise Comparisons
- Blog: A Framework for Proficiency-Aligned Grammar Practice in LLM-Based Dialogue Systems
- Blog: Language Models can Evaluate Themselves via Probability Discrepancy
- Blog: Stop Playing the Guessing Game! Target-free User Simulation for Evaluating Conversational Recommender Systems
- Blog: VC4VG: Optimizing Video Captions for Text-to-Video Generation
Showing 60 of 704 papers.
History & citation
Current · Permanent link
- Robustness and Bias in Large Language Model Evaluators merged into this direction (72% overlap)
- Reliability and Bias in LLM-as-a-Judge Frameworks merged into this direction (66% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_llm_as_a_judge_bias_a4282721,
title = {LLM-as-a-judge bias},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/llm-as-a-judge-bias/rdr_a4282721f5bc4b8d9c81448f5cf6e807},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_llm_as_a_judge_bias_a4282721_v1,
title = {LLM-as-a-judge bias},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/llm-as-a-judge-bias/rdr_a4282721f5bc4b8d9c81448f5cf6e807/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}