TL;DR
Voting Advice Applications (VAAs) help millions choose candidates, but their recommendations hinge on a "hidden" mathematical assumption: the numerical distance between Likert-scale options like "Agree" and "Strongly Agree." This paper demonstrates that varying this arbitrary spacing can change over 37% of top candidate recommendations and potentially shift enough parliamentary seats to decide an election.
What Problem Does This Paper Address?
In multi-party democracies, voters often use VAAs to find candidates who share their policy preferences. These tools typically ask users and candidates to respond to statements on a 5-point Likert scale (Strongly Disagree to Strongly Agree). To calculate a "match," the VAA must convert these qualitative labels into numbers.
Most developers assume "equidistance"—assigning values like 1, 2, 3, 4, and 5. However, there is no empirical evidence that people perceive the gap between "Neutral" and "Agree" as equal to the gap between "Agree" and "Strongly Agree." The authors investigate how this seemingly minor technical choice—the numerical spacing of intensity—biases recommendations toward certain types of candidates and parties.
Methods
The researchers audited the Kandidattest, a major Danish VAA. They introduced a parameter, $\alpha$, to represent the distance between a moderate stance (Agree/Disagree) and a strong stance (Strongly Agree/Disagree), while fixing the distance between "Neutral" and a moderate stance at 1.
Key methodological steps included:
- Proxy Voter Generation: Since real user data was unavailable for privacy reasons, the authors created 10,000 "proxy voters" by sampling real candidate responses and adding stochastic "noise" via a Markov chain. This preserved realistic correlations between different political issues.
- Sensitivity Analysis: They reran the matching algorithm across various values of $\alpha$ (from 0.25 to 3.0) to see how recommendations deviated from the baseline ($\alpha=1$).
- Quantifying Impact: They measured "matches" (aggregate counts for parties) and "switches" (how often an individual's top recommendation changed).
Datasets
- Candidate Responses: A dataset of responses from 858 unique candidates across 12 parties for the 2026 Danish General Election, provided by the media outlet Altinget.
- Election Results: National vote counts used to weight the sampling of proxy voters so they reflected the actual Danish electorate.
- Gender Data: Candidates were hand-labeled with binary gender classes to study distributional biases.
Key Insights and Ideas
- The "Intensity Dial": The parameter $\alpha$ acts as a dial for how much the algorithm punishes intensity mismatches. When $\alpha$ is low, the algorithm cares mostly about which side of an issue you are on. When $\alpha$ is high, it heavily penalizes a candidate for being "Strongly Agree" if the voter is only "Agree."
- Systematic Bias Toward Stance: Low $\alpha$ values systematically favor candidates who take "strong" stances. Conversely, high $\alpha$ values favor moderate candidates.
- Hidden Gender Bias: The study found that male candidates in the Danish dataset were more likely to choose "Strongly (Dis)agree" (42.7%) than female candidates (40.6%). Consequently, a low $\alpha$ setting inadvertently boosts recommendations for male candidates.
- Electoral Consequences: By combining their results with survey data on how many voters actually follow VAA advice, the authors estimate that changing $\alpha$ could shift between 4 and 13 seats in the 175-seat Danish parliament—a margin often larger than what decides which coalition forms the government.
Key Results
- Recommendation Volatility: Doubling the distance ($\alpha=2$) changed the top candidate match for 37.7% of voters and the top party match for 21.9%.
- Party-Level Swings: Some parties are highly sensitive to this parameter. For example, the party Alternativet received 19.6% more matches at low $\alpha$ values compared to the baseline.
- Gender Shifts: While the effect size was smaller (around 1-2%), high $\alpha$ values consistently favored female candidates, while low $\alpha$ favored males.
Limitations and Future Work
The authors acknowledge several limitations:
- Synthetic Data: The study relies on proxy voters rather than real user data. While their Markov chain method is more realistic than random sampling, it still assumes voters behave similarly to candidates.
- Symmetry Assumptions: The model assumes the same $\alpha$ applies to both ends of the scale and across all 29 policy questions. In reality, voters might perceive distances differently depending on the specific topic (e.g., economic vs. social issues).
- Simplified Matching: The audit ignored the "importance" weights users can assign to specific questions, which might interact with Likert spacing in complex ways.
Future research could focus on developing methods to infer perceived distances directly from user response data rather than assuming a fixed numerical scale.
Summary updated