RubriX: A Fast, Fine-Tunable Non-Autoregressive Rubric Judge

Balamurugan Paramasivam, Sudalai RajkumararXiv 2026 · arXiv

TL;DR

RubriX is a specialized, non-autoregressive model designed to act as a "rubric judge" for evaluating AI responses. By using a contrastive bi-encoder architecture instead of standard text generation, it achieves near-constant latency (~28ms) regardless of rubric length, making it up to 18.8x faster than traditional LLM-as-a-judge methods while maintaining competitive accuracy.

What Problem Does This Paper Address?

Evaluating AI systems often involves "LLM-as-a-judge," where a large model like GPT-4 is prompted with a response and a multi-level rubric (e.g., "Score this from 1-5 based on these criteria"). While flexible, this approach has three major drawbacks:

  1. Cost and Latency: Every judgment requires a full generative LLM call. As rubrics get longer or more detailed, the number of tokens the model must process grows, increasing both time and money spent.
  2. Context Limits: Very detailed rubrics can eventually exceed a model's context window.
  3. Brittleness: Generative scores are sensitive to prompt wording, decoding settings, and parsing errors.

The authors propose RubriX to move away from expensive text generation toward a more efficient, discriminative scoring method.

Methods

RubriX adopts an architecture inspired by Contrastive Language Models (CLM). Instead of reading the rubric and response together, it uses a bi-encoder setup:

  • Bi-Encoder Architecture: The input (conversation + response + criterion) and each rubric level description are encoded separately using a shared Qwen3-8B backbone adapted with LoRA.
  • Projection Heads: Two small multi-layer perceptrons (MLPs) project these embeddings into a shared 512-dimensional space.
  • Scoring Mechanism: The model calculates the temperature-scaled cosine similarity between the input embedding and each level embedding. A softmax over these similarities provides a probability distribution across the rubric levels (see Figure 1).
  • Ordinal Training: Unlike standard classification, rubric levels are ordered. The authors combine standard cross-entropy loss with Ranked Probability Score (RPS), which penalizes the model more for being "far off" (e.g., predicting a 1 when the answer is 5) than for being "close" (predicting a 4). They use an adaptive weighting scheme to balance these losses automatically during training.
Figure 1: The RubriX scorer. The input and each level description are encoded separately by the same encoder, projected by two heads, and compared by temperature-scaled cosine similarity. A softmax over the rubric’s own K levels gives a probability for each level. Level embeddings do not depend on the input, so they can be computed once per rubric and cached.

Datasets

The model was trained on a mix of 34,261 examples across 2,284 distinct criteria:

  • Public Data: Capped samples from Feedback Collection, HelpSteer, UltraFeedback, and ASAP 2.0.
  • Proprietary Data: 8,039 synthetic examples generated by an LLM and filtered by human-blind re-grading to ensure high quality.

Evaluation was conducted on six human-labelled benchmarks, including HealthBench, BiGGen, ELLIPSE, SummEval, JudgmentBench, and QAGS.

Key Insights and Ideas

  • The Power of Caching: Because the level descriptions are encoded separately from the response, their embeddings can be computed once and cached. For production systems using the same rubric for thousands of responses, the marginal cost of the rubric drops to zero.
  • Non-Autoregressive Efficiency: By treating evaluation as a matching task rather than a generation task, RubriX avoids the "KV cache" bottleneck of generative models.
  • Adaptive Loss Balancing: The authors found that RPS is much smaller in magnitude than cross-entropy. Their simple adaptive weighting (using exponential moving averages of loss magnitudes) allows the model to learn ordinal relationships without tedious hyperparameter tuning.

Key Results

  • Superior Generalization: On held-out rubrics (criteria never seen during training), RubriX achieved 75.4% exact accuracy, significantly outperforming Jev (57.1%) and a zero-shot Qwen3-8B judge (49.8%) (see Figure 2).
  • Massive Speedups: With cached embeddings, RubriX's latency remains flat at ~28ms. For a rubric with 3,000 tokens per level, RubriX is 7.7x faster than Jev and 18.8x faster than a Qwen3-8B judge.
  • Benchmark Performance: RubriX is highly competitive on human benchmarks, staying within 6 percentage points of the leading specialized model (Jev) on most sets and outperforming G-Eval (GPT-3.5) across the board.
Figure 2: Held-out test set: 6,749 judgments on rubrics whose criteria and level descriptions never appear in training.

Limitations and Future Work

  • Separate Encoding Bottleneck: Because the response and rubric levels never "meet" in the attention layers of the transformer, the model might struggle with tasks requiring extremely fine-grained factual cross-referencing between the response and specific rubric constraints.
  • Model Capacity: The current 8B parameter size may limit generalization to entirely unfamiliar rubric styles. The authors suggest that scaling to larger base models is a clear next step.
  • Fine-Tuning Dependence: RubriX performs best when the target rubrics are similar to its training data. For completely novel rubric formats, general-purpose generative judges might still hold an edge.
  • Future Directions: The authors plan to explore rubrics with hundreds of levels and domain-specific fine-tuning.

Summary updated

Abstract

Rubric-based evaluation asks which level of a rubric a response meets. It is usually done by prompting a large language model (LLM) as a judge, which needs a generative call per judgment and reads every level description, so its cost grows with the rubric. We present RubriX, a rubric judge whose architecture is inspired by contrastive language models (CLM): a LoRA-adapted encoder embeds the input and each level description, and a softmax over scaled cosine similarities gives a probability for every level without text generation. Trained on public rubric data and proprietary synthetic data, RubriX reaches 75.4% exact accuracy on held-out rubrics, against 57.1% for TypeSafe's Jev and 49.8% for a Qwen3-8B judge. On eight human-labelled benchmark sets it is within 6 percentage points of Jev's accuracy on most of them, while Jev keeps a larger lead on two and ranks responses best on average; against the results reported in the G-Eval paper, RubriX outperforms G-Eval with GPT-3.5 on every SummEval dimension and both QAGS subsets. With level embeddings cached, RubriX's latency stays at about 28 ms per judgment as level descriptions grow from 10 to 3,000 tokens, while Jev's (approximate network time subtracted) rises from 44 ms to 211 ms and the Qwen3-8B judge's from 32 ms to 516 ms, making RubriX 7.7x faster than Jev and 18.8x faster than the Qwen3-8B judge at 3,000 tokens per level.
BibTeX
@misc{paramasivam2026rubrix,
  author = {Balamurugan Paramasivam and Sudalai Rajkumar},
  title = {{RubriX: A Fast, Fine-Tunable Non-Autoregressive Rubric Judge}},
  year = {2026},
  eprint = {2610.34875},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.34875},
}
Figure