Language models are now commonly deployed behind structured interfaces. A content-moderation API returns {"label": "hateful"}, and an agent fills a JSON form with a verdict. The tool that guarantees such outputs parse is grammar-constrained decoding: at every step it blocks any token that would break a formal grammar or schema. That guarantee is usually treated as neutral plumbing. We argue it is not. Two schema choices decide which human perspective the output reflects. The first is a single-choice label field, the enum ("pick exactly one of these strings"). The second is the order in which fields are generated. On items where human annotators genuinely disagree, a model may internally spread probability across labels much as annotators split their votes. A greedy constrained decode then rounds that spread to the majority label, every time. On items where the context is too thin to answer, putting the answer field before the reasoning field removes the step at which the model might notice it lacks information. That is exactly the condition under which models fall back on stereotypes.
The proposal has two parts. First, we measure how much this "majority collapse" and stereotype fallback are caused by the decoding schema itself, separate from the underlying model. We do this on benchmarks whose labels come with documented disagreement, including one where the disagreement splits along annotator income. Second, we test a cheap fix. The grammar automaton that enforces the schema can also compute the model's exact probability over every valid filled-in schema. The decoder can return that distribution, or a "contested" value when it is spread out, instead of a forced single label. The claim is that much of what looks like a model erasing minority annotator views is introduced at decode time, and can be removed at decode time without retraining.
Core Claim
On disagreement-dense items, emitting an enum label under greedy grammar-constrained decoding raises how often the majority label is produced, beyond the rate at which annotators actually chose it. Answer-first field order additionally increases stereotype-consistent answers in under-informative contexts. Reading out the exact probability over valid schema completions, rather than a single argmax, recovers annotator-level vote distributions better than three alternatives: free-text generation, verbalized confidence, and the soft-label and per-annotator classifiers that prior work found inadequate.
We define amplification on an item set $D$ (for example, the items where annotators disagree) as
$$A(D) = \frac{1}{|D|}\sum_{x \in D} \Big[\, \mathbb{1}\{\hat y(x) = m(x)\} - v_{m(x)}(x) \,\Big].$$
Here $\hat y(x)$ is the label the system emits for item $x$, $m(x)$ is the annotators' majority label, and $v_{m(x)}(x)$ is the fraction of annotators who chose it. $A>0$ means the system sides with the majority more often than annotators themselves did. For a distribution-reporting system, we replace the indicator with the probability the system assigns to $m(x)$.
How This Relates to Prior Work
Several papers motivate the gap. The analysis in Majority Vote Silences Minority Values shows that HateXplain disagreement is structural rather than noise. Of all disagreements, 42.6% fall at the hate/offensive boundary, and annotators often highlight identical tokens yet choose different labels. That paper also trained BERT classifiers on hard labels, soft labels, and per-annotator heads. All of them stayed about 70% aligned with the majority vote, and the per-annotator model reached only 0.245 accuracy on offensive disagreement cases. That result constrains our design: training-time changes alone do not preserve minority views. It also leaves autoregressive, schema-constrained generation untested. BBQ shows that in ambiguous contexts roughly 77% of a QA model's errors follow stereotypes. Uncovering Stereotypes in Large Language Models shows that refusal falls and stereotyping rises as task load increases. Together these suggest that removing deliberation is likely to increase stereotype fallback. Born with a SilverSpoon? supplies the demographic signal HateXplain lacks: ethical dilemmas labeled separately by annotators earning over \$200k and under \$40k. It reports that models side with the higher-income view. All of its evaluation is free-form prompting, never structured outputs.
On the decoding side, Let Me Speak Freely? is the closest work. It measures a "reasoning tax" from answer-first JSON: GPT-3.5-Turbo drops from about 56% to 25% on Last Letter Concatenation. It also evaluates stereotype detection, but only as static classification scored against a single gold label. It also reports a finding that appears to contradict us: JSON mode can act as a "denoiser" that raises classification accuracy, from 41.6% to 60.3% for Gemini-1.5-Flash on DDXPlus. We take this head on. If the gold label is the majority vote, pushing outputs toward the majority would raise accuracy while worsening distributional fidelity. Our metrics separate the two, and if the denoising effect also appears under distributional metrics, our claim weakens.
Mitigating Bias in Locally Constrained Decoding via Tractable Proposals supplies part of the method. It shows that token-by-token masking (local constrained decoding) skews the sampled distribution away from the model's true conditional over valid strings. It also builds tensorized automata, with about 1.1x overhead, that account for whether a valid completion is still reachable (global constrained decoding). Its notion of bias is purely statistical. We reuse its global decoder to obtain exact schema marginals and ask whether that statistical bias has a social direction. Validating Large Language Models with ReLM used regular-expression automata to audit gender–occupation bias in GPT-2. It found that looking only at the standard tokenization of a string underestimates bias. We borrow its practice of summing over all token paths, but apply it to production decoding of instruction-tuned models rather than to offline probing. Flexible and Efficient Grammar-Constrained Decoding (GREATGRAMMA, 5–32 ms per-token masking) provides the engine for our local-masking condition. DARC is the nearest disagreement-aware decoder. It reranks complete unconstrained responses using a risk penalty based on sensitivity to input perturbations. It has no grammar, makes no token-level intervention, and uses no demographic labels.
Research Program
We build one evaluation harness with three tasks. Each is posed as a schema-filling problem in which every item carries a human vote distribution.
The first task uses HateXplain posts. The schema contains a label enum (hatespeech, offensive, normal), a target-community field, and a rationale field. The three-annotator votes give the reference distribution. The second task uses SilverSpoon Part 1 dilemmas with a verdict enum, so that agreement can be scored separately against the high-income and the low-income annotators. The third task uses BBQ ambiguous and disambiguated contexts with an answer enum that includes "unknown." BBQ has no annotator split, so it serves purely to measure stereotype amplification.
We cross two schema factors. The first is field order: reasoning-first versus answer-first. The second is the readout. We compare four readouts. The first is greedy constrained decoding. The second is sampling under local masking. The third is the exact marginal over enum values, computed with the global automaton by summing over all token paths and over the remaining fields. The fourth is a schema-only change that adds a "contested" enum value; the decoder emits it when the exact marginal's entropy exceeds a threshold tuned on a held-out split.
A careful reader will notice something. For a lone enum with a fixed prefix, the exact marginal reduces to scoring each label string by teacher forcing; no grammar engine is needed. The automaton becomes necessary when the label is jointly constrained with other fields. Examples are the target-community field, length-bounded rationales, and the label sitting after a reasoning field generated under the constraint. In those cases, local masking's mis-weighting compounds across fields. We will report both the simple-scoring case and the joint case, so the contribution of grammar machinery is visible rather than assumed.
Models are open-weight instruction models, Llama-3.1-8B-Instruct and Qwen2.5 as in DARC, plus one larger model where compute allows. Metrics are the following:
- Amplification $A$ on disagreement items.
- Jensen–Shannon divergence from the annotator vote distribution.
- Recall of the minority label on disputed items.
- Majority-label accuracy, reported so that the "denoiser" effect is visible.
- Per-income-group Cohen's kappa on SilverSpoon.
- BBQ bias score and accuracy in both ambiguous and disambiguated contexts.
- Parse validity and wall-clock overhead.
Baselines
The naive reference point is unconstrained natural-language answering, with the label extracted by regular expression. It is given the same prompt and the same models. It tells us what the model does before any schema intervenes, and the schema effects are measured as differences from it.
The deployment default is greedy decoding under GREATGRAMMA-style local masking with an answer-first schema (Flexible and Efficient Grammar-Constrained Decoding). We include it because it is what practitioners actually run. Beating it means lower $A$ and lower divergence at equal parse validity.
The strongest existing method for recovering disagreement is the per-annotator and soft-label BERT family from Majority Vote Silences Minority Values. The reported numbers are about 58% accuracy on disagreement posts and about 70% majority alignment for all variants. The per-annotator model reaches 0.245 accuracy on offensive disagreement cases, and its confidence on boundary errors is 0.495, compared with 0.710 for the hard-label model. We will retrain these models on our exact HateXplain splits rather than compare against the published numbers. Beating them requires lower majority alignment on disputed items without losing accuracy on agreed items. The paper reports no distributional metric, so for divergence we have no reported result on Lacuna and will compute it ourselves.
The strongest decoding-side competitor is the two-step pipeline from Let Me Speak Freely?. The model reasons in free text, and a second pass then formats the answer. That paper shows it preserves reasoning, for example on Last Letter Concatenation, where answer-first JSON drops GPT-3.5-Turbo from roughly 56% to 25%. We run it with the same models, and the formatting pass gets a matched token budget. We also include DARC-style risk-penalized Best-of-K reranking over unconstrained candidates (DARC), with K matched to our compute. DARC reports its gains as reductions in CVaR10% on MT-Bench and AlpacaEval; it reports no number on our tasks, so there is no reported result on Lacuna for DARC on HateXplain, BBQ, or SilverSpoon.
The key ablation is the 2x2 crossing of field order with greedy-versus-marginal readout. If the marginal readout helps only in the reasoning-first condition, then the gain comes from deliberation, not from distribution reporting. Swapping local for global masking at a fixed readout isolates the contribution of the grammar algorithm itself.
Why not just…?
Why not just ask the model to state its confidence or a label distribution in text? This is the most serious threat, because it needs no decoder changes at all. We include verbalized distributions, constrained to a valid probability field, as a baseline under identical prompts. The file contains no evidence on how well verbalized confidence matches annotator splits. If it equals the exact marginal in divergence and minority recall, the fix component of this proposal reduces to a prompting recommendation. The measurement of schema-induced amplification would still stand.
Why not just use the two-step natural-language-then-format pipeline? Let Me Speak Freely? shows it avoids the reasoning tax, and it may also avoid stereotype fallback on BBQ. However, it still emits a single label, so we expect it to fix field order but not majority collapse. The 2x2 design shows this directly. It also costs a second generation pass, which we will report as latency.
Why not just sample K unconstrained answers and count the labels? Counting estimates the same distribution as the exact marginal. Its error, however, shrinks only as about $1/\sqrt{K}$, the standard Monte Carlo rate. With three-way labels and ten-point differences between groups, that requires many samples per item. We plot divergence against K to locate the crossover with the single-pass exact marginal. If small K suffices, we will say so.
Why not fix it upstream with diverse annotators or soft-label fine-tuning? We agree this is worthwhile. But Majority Vote Silences Minority Values found that soft-label and per-annotator training still left models about 70% majority-aligned. Moreover, a deployed schema can re-collapse any distribution a model learns, so decode-time behavior has to be measured regardless.
Decisive Checks
The proposal's measurement claim fails if $A$ under greedy constrained decoding is no larger than under unconstrained answering on HateXplain disagreement items. It also fails if answer-first order does not raise the BBQ ambiguous-context bias score relative to reasoning-first. The fix claim fails if the exact-marginal readout does not reduce divergence from the vote distribution. It also fails if, on SilverSpoon, it does not close the kappa gap between high-income and low-income annotators relative to greedy decoding.
A secondary check addresses the denoising effect. If majority-label accuracy rises under JSON while divergence also falls, the denoising is real and not merely majority collapse, and we will report it that way. Finally, we will report parse validity, which must stay at 100% for all constrained conditions, and the overhead of global decoding, which prior work puts near 1.1x. These confirm that the fix has no practical cost.
References
- Majority Vote Silences Minority Values: Annotator Disagreement at the Hate/Offensive Boundary in HateXplain
- BBQ: A hand-built bias benchmark for question answering
- Uncovering Stereotypes in Large Language Models: A Task Complexity-based Approach
- Born with a SilverSpoon? Investigating Socioeconomic Bias in LLMs
- Let Me Speak Freely? A Study On The Impact Of Format Restrictions On Large Language Model Performance
- Mitigating Bias in Locally Constrained Decoding via Tractable Proposals
- Validating Large Language Models with ReLM
- Flexible and Efficient Grammar-Constrained Decoding
- DARC: Disagreement-Aware Alignment via Risk-Constrained Decoding