When building datasets for objective tasks—like identifying a stop sign or transcribing audio—disagreement among human annotators usually implies error. In those cases, taking a majority vote is an elegant way to filter out noise and uncover a single, objective ground truth.
However, as machine learning tackles highly subjective domains like hate speech detection, cultural nuances, humor, and moral values, this consensus-driven approach breaks down. When annotators disagree on whether a tweet is offensive or whether an image is culturally representative, the disagreement is rarely a mistake. Instead, it reflects valid differences in human background, ideology, and culture.
As Cabitza et al. argue in Toward a Perspectivist Turn for Predictive Computing, forcing a single "gold standard" label via majority voting systematically silences valid minority viewpoints and specialized interpretations. This reductionist approach creates a fundamental mismatch between the pluralistic nature of human society and the monolithic consensus demanded by standard machine learning pipelines.
The Optimization Mechanism of Erasure
To understand why majority-vote aggregation fails, we have to look at how it shapes the optimization landscape during training.
When a dataset collapses diverse human labels into a single majority-rule consensus, it doesn't just ignore minority opinions—it actively forces the model to un-learn them. In a standard supervised learning setup, the loss function penalizes any prediction that deviates from the aggregated ground truth. If 60% of an annotation pool labels a phrase as "acceptable" and 40% labels it as "offensive," a majority-vote pipeline sets the ground truth to "acceptable." During training, if the model attempts to capture the nuanced perspective of the 40% minority, it is penalized by the loss function.
This problem is severely amplified in modern alignment techniques like Reinforcement Learning from Human Feedback (RLHF). As Casper et al. point out in Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback, RLHF compresses complex, pluralistic human values into a single scalar reward signal. This mathematical flattening ensures that the preferences of the majority demographic dominate the reward model.
In No Preference Left Behind: Group Distributional Preference Optimization, Yao et al. demonstrate the mathematical consequence of this design: standard preference optimization objectives treat conflicting data from minority demographics as statistical noise around a central mean. As a result, the reward margins for minority opinions consistently decrease over the course of training. The model is effectively optimized to treat the dominant cultural norm as the only norm, resulting in what researchers call minority perspective erasure.
This erasure introduces severe evaluation gaps. When models are evaluated against benchmarks that rely on majority-vote verification, the metrics fail to capture how the model harms or misrepresents sub-cultures. As Hall demonstrates in Towards Geographic Inclusion in the Evaluation of Text-to-Image Models, using majority-vote consensus to evaluate generated content treats subjective cultural preferences as a binary pass/fail metric. This masks deep cross-cultural disagreements, reinforcing a single, often Western-centric standard of correctness while penalizing models that align with valid regional perspectives.
Moving to Perspectivist Architectures
To prevent the erasure of minority viewpoints, a growing body of research focuses on "perspectivist" machine learning. The core intuition is simple: preserve the distribution of human opinions rather than collapsing it.
Traditional Pipeline:
[Individual Labels] ──> (Majority Vote) ──> [Single Gold Label] ──> Loss Minimization
Perspectivist Pipeline:
[Individual Labels] ──> (Distribution Modeling) ──> [Label Probability Vector] ──> Multi-Perspective Loss
└─> [Annotator Embeddings] ──────┘
Instead of training a model to output a single categorical label, models can be designed to predict the full probability distribution of human judgments. In Disagreement Matters: Preserving Label Diversity by Jointly Modeling Item and Annotator Label Distributions with DisCo, Weerasooriya introduces a framework that jointly models text features alongside individual annotator behavior. By learning the soft distribution of labels, the model retains the knowledge that a specific item is highly contested, preserving demographic-specific interpretations.
Another powerful mechanism is to explicitly model the annotators themselves within the network architecture. Deng et al. implement this in You Are What You Annotate: Towards Better Models through Annotator Representations by introducing learnable annotator embeddings. When training on a subjective task, the model takes both the text input and an annotator ID. This allows the network to map personal, cultural, or ideological backgrounds into a continuous latent space.
Instead of treating a disagreement between two annotators as a conflicting gradient update that cancels itself out, the model learns why they disagree based on their positions in the embedding space. Similarly, Heinisch et al. show in Architectural Sweet Spots for Modeling Human Label Variation by the Example of Argument Quality: It's Best to Relate Perspectives! that leveraging recommender-system techniques to relate different annotator perspectives to one another allows the model to generalize across sparse datasets without defaulting to a biased average.
Pluralistic Alignment and Multi-Agent Debate
For large language models, preserving minority perspectives requires rethinking inference and alignment strategies. If a model must generate a single response to a user, how can it avoid defaulting to a majority-rule outcome that marginalizes underrepresented groups?
One approach is to factorize alignment into distinct cultural or demographic lenses. Chen et al. propose this in SPICA: Retrieving Scenarios for Pluralistic In-Context Alignment, steering models toward the specific values and norms of different demographic groups at inference time using group-informed context.
When a single response is required but the training data contains deeply conflicting values, optimization frameworks must be explicitly designed to protect the minority. Chakraborty et al. address this in MaxMin-RLHF: Alignment with Diverse Human Preferences by learning a mixture of distinct reward models representing different user groups. Instead of maximizing the average reward across all users (which favors the majority), the objective function maximizes the minimum reward across all groups, ensuring that the model's behavior remains acceptable even to the most marginalized or non-mainstream perspectives.
Alternatively, the conflict between different labeling guidelines can be resolved dynamically through language itself. In PREDICT: Multi-Agent-based Debate Simulation for Generalized Hate Speech Detection, Park bypasses static label aggregation entirely by assigning different language model agents the specific perspectives of different annotation frameworks. By forcing these agents to debate ambiguous text, the system surfaces the underlying social, historical, and linguistic contexts that a single majority vote would completely erase.
Why Disagreement Works as a Signal
The fundamental reason this research direction works is that human disagreement is not noise; it is a dense diagnostic signal.
When we force a consensus label, we discard the variance in our data. Yet that variance contains critical information about linguistic ambiguity, task difficulty, and cultural boundaries. As Jinadu et al. demonstrate in Noise Correction on Subjective Datasets, introducing a explicit subjectivity parameter allows a model to mathematically separate genuine human disagreement from actual random error, leading to far more robust representations.
Even within a model's own internal generations, minority outputs hold value. In Mirror-Consistency: Harnessing Inconsistency in Majority Voting, Huang et al. show that when an LLM generates multiple responses, the dissenting, minority outputs frequently highlight specific reasoning traps or areas of deep model uncertainty.
By shifting from a paradigm of consensus-driven erasure to one of distribution preservation, machine learning models stop treating human diversity as a problem to be solved and start using it as a signal to be learned.