Hidden Distances: How Likert-Scale Spacing Shapes Algorithmic Voting Advice

Thomas Broholt Knudsen, Christian Høst-Madsen, Vedran Sekara, Jonas L. JuularXiv 2026 · arXiv

TL;DR

Voting Advice Applications (VAAs) help millions choose candidates, but their recommendations hinge on a "hidden" mathematical assumption: the numerical distance between Likert-scale options like "Agree" and "Strongly Agree." This paper demonstrates that varying this arbitrary spacing can change over 37% of top candidate recommendations and potentially shift enough parliamentary seats to decide an election.

What Problem Does This Paper Address?

In multi-party democracies, voters often use VAAs to find candidates who share their policy preferences. These tools typically ask users and candidates to respond to statements on a 5-point Likert scale (Strongly Disagree to Strongly Agree). To calculate a "match," the VAA must convert these qualitative labels into numbers.

Most developers assume "equidistance"—assigning values like 1, 2, 3, 4, and 5. However, there is no empirical evidence that people perceive the gap between "Neutral" and "Agree" as equal to the gap between "Agree" and "Strongly Agree." The authors investigate how this seemingly minor technical choice—the numerical spacing of intensity—biases recommendations toward certain types of candidates and parties.

Methods

The researchers audited the Kandidattest, a major Danish VAA. They introduced a parameter, $\alpha$, to represent the distance between a moderate stance (Agree/Disagree) and a strong stance (Strongly Agree/Disagree), while fixing the distance between "Neutral" and a moderate stance at 1.

Fig. 1. Overview of the experimental setup. a, Proxy voters are created from the candidates: 1,000 candidates are sampled with replacement per constituency, proportional to the parties’ vote shares, giving 10,000 proxy voters. Noise is then applied to their answers (grey cells mark changed answers). SA, A, D and SD denote “strongly agree”, “agree”, “disagree” and “strongly disagree”, which are mapped to numerical values as shown, with 𝛼setting the distance between the strong stances options and their moderate neighboring options and the neutral option hidden in the test. b, Each proxy voter is matched against all candidates in their constituency using the Manhattan distance across the 29 issues, converted to a relative agreement in [0,1], and the top 10 candidates are returned. c, Changing how responses are encoded using the parameter 𝛼. Values 𝛼< 1 compress the distance between extreme ends of the scale, 𝛼= 1 returns the original scale used by the VAA developers, while 𝛼> 1 elongates the scale. d, Top matches are counted per party for each 𝛼-value. In the example, one proxy voter moves from party B to party A when the response scale with 𝛼= 2 is used compared to the baseline of 𝛼= 1 (dashed = lost, grey = gained). e, Advice is compared to the 𝛼= 1 baseline used in the Kandidattest. A switch is counted whenever the set of top-matched candidates (or parties) differs from the baseline set.

Key methodological steps included:

  • Proxy Voter Generation: Since real user data was unavailable for privacy reasons, the authors created 10,000 "proxy voters" by sampling real candidate responses and adding stochastic "noise" via a Markov chain. This preserved realistic correlations between different political issues.
  • Sensitivity Analysis: They reran the matching algorithm across various values of $\alpha$ (from 0.25 to 3.0) to see how recommendations deviated from the baseline ($\alpha=1$).
  • Quantifying Impact: They measured "matches" (aggregate counts for parties) and "switches" (how often an individual's top recommendation changed).

Datasets

  • Candidate Responses: A dataset of responses from 858 unique candidates across 12 parties for the 2026 Danish General Election, provided by the media outlet Altinget.
  • Election Results: National vote counts used to weight the sampling of proxy voters so they reflected the actual Danish electorate.
  • Gender Data: Candidates were hand-labeled with binary gender classes to study distributional biases.
Visualization of proxy voters in Principal Component (PC) space compared to real candidates.

Key Insights and Ideas

  • The "Intensity Dial": The parameter $\alpha$ acts as a dial for how much the algorithm punishes intensity mismatches. When $\alpha$ is low, the algorithm cares mostly about which side of an issue you are on. When $\alpha$ is high, it heavily penalizes a candidate for being "Strongly Agree" if the voter is only "Agree."
  • Systematic Bias Toward Stance: Low $\alpha$ values systematically favor candidates who take "strong" stances. Conversely, high $\alpha$ values favor moderate candidates.
  • Hidden Gender Bias: The study found that male candidates in the Danish dataset were more likely to choose "Strongly (Dis)agree" (42.7%) than female candidates (40.6%). Consequently, a low $\alpha$ setting inadvertently boosts recommendations for male candidates.
  • Electoral Consequences: By combining their results with survey data on how many voters actually follow VAA advice, the authors estimate that changing $\alpha$ could shift between 4 and 13 seats in the 175-seat Danish parliament—a margin often larger than what decides which coalition forms the government.

Key Results

  • Recommendation Volatility: Doubling the distance ($\alpha=2$) changed the top candidate match for 37.7% of voters and the top party match for 21.9%.
  • Party-Level Swings: Some parties are highly sensitive to this parameter. For example, the party Alternativet received 19.6% more matches at low $\alpha$ values compared to the baseline.
  • Gender Shifts: While the effect size was smaller (around 1-2%), high $\alpha$ values consistently favored female candidates, while low $\alpha$ favored males.
Switch frequencies and party-level match changes across different $\alpha$ values.

Limitations and Future Work

The authors acknowledge several limitations:

  • Synthetic Data: The study relies on proxy voters rather than real user data. While their Markov chain method is more realistic than random sampling, it still assumes voters behave similarly to candidates.
  • Symmetry Assumptions: The model assumes the same $\alpha$ applies to both ends of the scale and across all 29 policy questions. In reality, voters might perceive distances differently depending on the specific topic (e.g., economic vs. social issues).
  • Simplified Matching: The audit ignored the "importance" weights users can assign to specific questions, which might interact with Likert spacing in complex ways.

Future research could focus on developing methods to infer perceived distances directly from user response data rather than assuming a fixed numerical scale.

Summary updated

Abstract

Voting Advice Applications (VAAs) are algorithmic recommenders that match voters with political candidates on policy preferences. They are widely adopted across many multi-party democracies; in Denmark, 62\% of eligible voters used a VAA ahead of the 2022 general election, and 45\% of VAA users reported following its advice. VAAs typically ask voters and candidates to respond to policy statements on a Likert scale, from ``strongly disagree'' to ``strongly agree'', and recommend candidates whose responses are closest to the voter's. Computing these distances requires assigning numbers to response options, a design choice without an empirical basis, since it is unknown how people perceive the distances between options. Here we show that this seemingly technical choice substantially changes whom voters are matched with. We parameterize the distance between ``(dis)agree'' and ``strongly (dis)agree'' relative to the distance between ``neutral'' and ``(dis)agree'' as $α$, and quantify how it changes recommendations for a popular Danish VAA. We find that moving from equidistant options to doubling this distance can change $37.7\%$ of top candidate matches and $21.9\%$ of top party matches, although no answers change. Low $α$-values favor candidates with strong positions, with some parties receiving up to $ 19.6\%$ more recommendations, while high $α$-values favor moderates. Similarly, female candidates receive fewer recommendations at low $α$ and more at high $α$-values. Our findings demonstrate that undisclosed quantification choices in algorithmic recommenders can carry considerable electoral consequences, and they call for transparency and sensitivity analysis in VAA design. As Likert-based matching also underpins dating platforms and roommate allocation systems, these implications extend well beyond elections.
BibTeX
@misc{knudsen2026hidden,
  author = {Thomas Broholt Knudsen and Christian Høst-Madsen and Vedran Sekara and Jonas L. Juul},
  title = {{Hidden Distances: How Likert-Scale Spacing Shapes Algorithmic Voting Advice}},
  year = {2026},
  eprint = {2611.12313},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12313},
}
Figure