Continuous Score Estimation from Discrete Token Probabilities

36 concepts29 papers1 proposal

When we task a machine learning model with evaluation—whether grading an essay, assessing the beauty of an image, or judging the helpfulness of a chatbot response—we traditionally force it to make a discrete choice. We might ask a classifier to assign an integer from 1 to 5, or prompt a Large Language Model (LLM) to output a specific rating token.

However, this discrete framing introduces a fundamental bottleneck: quantization error. If a model evaluates an essay as sitting precisely halfway between a 4 and a 5, forcing a hard choice drops crucial nuance. This leads to high-variance scoring, frequent ties when ranking similar items, and an inability to capture the model's internal uncertainty. As noted by Xingyuan Ma et al. in Regression over Classification: Assessing Image Aesthetics via Multimodal Large Language Models, treating numerical scores as isolated words prevents models from naturally "feeling" the mathematical continuity of numbers.

The solution emerging across the field is to bypass the final discrete selection (the argmax) and instead look under the hood at the model's softmax logits. By treating the probability distribution over discrete tokens as weights, researchers can calculate a continuous expected value. This shift transforms rigid, peaky classifications into smooth, high-resolution metrics.


The Core Mechanism: From Argmax to Expected Value

To understand why this works, consider a model prompted to score a text generation on a scale from 1 to 3. Instead of reading the single token the model generates, we extract the raw probabilities assigned to the tokens "1", "2", and "3".

If Model A outputs a probability distribution of $[0.0, 0.1, 0.9]$ and Model B outputs $[0.0, 0.4, 0.6]$, a standard text-based extraction rounds both down to a discrete score of 3. By shifting to an expected value calculation, we compute a weighted average:

$$\text{Continuous Score} = \sum_{i} P(\text{token}_i) \times \text{Value}_i$$

Model A receives a continuous score of $2.9$, while Model B receives $2.6$. The evaluation immediately gains a higher resolution, allowing us to see that Model A is significantly closer to a perfect score than Model B.

This core mechanism has deep roots in early automated scoring systems. For instance, Suma Bhat et al. applied this to non-native speech evaluation in Shallow Analysis Based Assessment of Syntactic Complexity for Automated Speech Scoring, calculating the expected value across proficiency categories to maintain a granular metric even when speech recognition software introduced errors. Similarly, in non-intrusive speech quality assessment, Chunlei Zhang et al. demonstrated in MetricNet: Towards Improved Modeling For Non-Intrusive Speech Quality Assessment that taking the expected value over a range of predicted score bins smooths out individual bin errors, yielding a more stable and robust output.


Smoothing the LLM-as-a-Judge

With the widespread adoption of LLMs as evaluators, probability-weighted scoring has become a critical tool for aligning model outputs with human judgment.

In the G-EVAL framework, Yang Liu et al. combined Chain-of-Thought prompting with logit extraction in G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment. By taking the weighted average of score token probabilities, the authors successfully mitigated the low-variance problem inherent to discrete integer scoring, resolving ties between high-performing summaries. This strategy is mirrored in dialogue evaluation by Jinghan Jia et al. in Leveraging LLMs for Dialogue Quality Measurement and in topic model analysis by Alexander Miserlis Hoyle et al. in ProxAnn: Use-Oriented Evaluations of Topic Models and Document Clustering, where extracting the log-probabilities of ordinal tokens serves to capture the model's internal confidence.

Beyond breaking ties, preserving the model's internal uncertainty eliminates systemic logical glitches. Standard integer scoring often causes "grade inflation" and introduces contradictions where a judge might rate a response highly but rank it lower in a head-to-head matchup.


Extending the Paradigm: Text Levels, Decimals, and Pairwise Duels

The flexibility of probability-weighted scoring means the underlying tokens do not even need to be numerical digits. The technique can map qualitative text labels, multi-digit decimals, or relative preferences onto a continuous spectrum.

Qualitative Text Levels

Instead of forcing a model to reason about numbers, the authors of Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels, Haoning Wu et al., trained multimodal models to predict discrete text levels like "excellent" or "poor". By extracting the log probabilities of these target text tokens and calculating a weighted average via softmax, they derived a continuous quality score that achieved state-of-the-art generalization across multiple benchmarks.

High-Resolution Decimals

To achieve arbitrary precision without massive models, Yebin Lee et al. in FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model and Hyunjong Kim et al. in EXPERT: An Explainable Image Captioning Evaluation Metric with Structured Explanations calculated weighted sums of token probabilities at specific decimal positions. This smoothing technique allows smaller, open-source models to output high-resolution continuous scores that can outperform larger, closed-source models whose internal token probabilities are hidden behind an API.

Pairwise Preferences

When ranking documents, Jian Luo et al. developed a framework in PRP-Graph: Pairwise Ranking Prompting to LLMs with Graph Aggregation for Effective Text Re-ranking that extracts continuous confidence scores from softmax logits during pairwise comparisons rather than relying on discrete win/loss labels. This builds on an intuition established by Javier González et al. in Preferential Bayesian Optimization, which proved that aggregating pairwise win probabilities into a continuous score results in a smoother objective function, facilitating more effective numerical optimization.


Broad Utility Across Machine Learning Tasks

The power of softening discrete boundaries into continuous probabilities extends far beyond evaluation metrics, proving useful across search, domain adaptation, and statistical estimation:

By shifting the focus from what token a model ultimately chooses to the underlying distribution of what it considers, probability-weighted scoring unlocks a high-resolution, continuous window into a model's internal reasoning.

Key Foundational Work

Shallow Analysis Based Assessment of Syntactic Complexity for Automated Speech Scoring (2014) — contributed 1 concept elements to this direction.
A probabilistic feature-based fill-up for SMT (2014) — contributed 1 concept elements to this direction.
An individual-specific strategy for management of reference data in adaptive ensembles for person re-identification (2013) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_probability_weighted_scoring_db0328f3,
  title = {probability-weighted scoring},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/probability-weighted-scoring/rdr_db0328f3cba14ff487e2daf899afef14},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_probability_weighted_scoring_db0328f3_v1,
  title = {probability-weighted scoring},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/probability-weighted-scoring/rdr_db0328f3cba14ff487e2daf899afef14/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake