Shared Concepts, Personal Weights: Crossing the Unseen-Annotator Wall in Toxicity Prediction

People disagree about whether a comment is offensive, and a moderation model that learns one "gold" label hides that disagreement. Recent work that tries to model each annotator separately runs into a hard limit. When a large language model is fine-tuned to predict individual annotators' offensiveness and safety judgments, it does well on annotators it has already seen. On annotators held out entirely, it falls back to the accuracy of a model that reads only the text (Beyond Demographics). The authors trace this to a "proxy ID" effect: the model memorizes who the annotator is instead of learning how people differ.

We propose that most of this disagreement lives in a small, nameable place. Annotators largely agree on what is in a comment: whether it contains an insult, a threat, or an attack on an identity group, and whether a slur is used or only quoted. They differ in how much each of these matters to them. If that holds, a model can split the problem in two. A shared text encoder, trained on several tasks at once, predicts a short vector of human-readable concepts. Each annotator is then a small linear rule over those concepts, meaning one weight per concept. A new annotator is described by roughly ten numbers, which can be estimated from a handful of their labels, and those numbers read directly as "this person penalizes identity attacks heavily but tolerates profanity." The claim is concrete and falsifiable. On annotators never seen in training, this concept-weight model should beat black-box personalization when only a few calibration labels are available. It should also stay competitive on seen annotators, and it should make the reason for each individual judgment inspectable and correctable.

Core Claim

The four directions play different roles here. Content moderation is the application. Multi-task learning builds the shared concept layer. The concept bottleneck supplies the interpretable structure that makes personalization cheap. Leave-one-subject-out evaluation, applied to annotators, is the test that decides whether any of this works.

A concept bottleneck model maps an input $x$ to predicted concepts $\hat c = g(x) \in \mathbb{R}^k$, then maps those concepts to a label through a simple, transparent function (Concept Bottleneck Models). That work showed bottlenecks cost little accuracy on bird classification (19.9% versus 17.5% error), let humans fix predictions by editing concepts, and held up better under background shift. It used objective vision concepts with a single ground truth and never looked at subjective labels or held-out people. Our model gives each annotator $a$ their own transparent head:

$$p(y_{a} = 1 \mid x) = \sigma\left(w_a^\top \hat c(x) + b_a\right),$$

Here $y_a$ is annotator $a$'s binary judgment, $\sigma$ is the logistic function, and $w_a \in \mathbb{R}^k$ and $b_a$ are that annotator's concept weights and bias. For a new annotator we place a population prior on the weights, $w_a \sim \mathcal{N}(\mu, \Sigma)$, fitted across training annotators. We then compute the posterior from their first $m$ labels. With $k \approx 10$, a few labels already constrain $w_a$. A personalized black box has to locate the same person in a space with hundreds or thousands of dimensions, which is why we expect the gap to be largest at small $m$.

This idea connects to what calibration does for biosignals. In electromyography, subject-independent generalization is poor, but fine-tuning on 5% of a new subject's data raised accuracy by 44% on average (EMGBench). We treat that finding as motivation. The generalization wall may be the wrong target, and the right question is how few labels it takes to get over it, with the answer phrased in human terms.

Research Program

The concept layer is built by multi-task training of one encoder on several concept-supervision sources. The first source is six toxicity attributes: toxic, severe toxic, obscene, threat, insult, and identity attack. These are the same attribute heads that Tox-BART found more useful than general knowledge graphs for explaining implicit hate. There, the attributes were prepended as prompt text. Here, they become the only path to the label. The second source is a use-versus-mention concept trained on the counterspeech pairs of Gligoric et al.. That paper showed classifiers ranging from RoBERTa to GPT-4 over-censor counterspeech, and it reduced the problem with prompting rather than with a trainable, editable variable. The third source is target-group mention, taken from Civil Comments identity metadata.

Training all concept heads jointly is the multi-task step. Work on joint objectives suggests shared training produces representations that serve every head better than isolated pipelines (Situ et al., Yang et al.). Negative transfer across heterogeneous sources is a known risk, so we compare against single-source concept encoders. We also borrow the soft-label auxiliary head of Fornaciari et al. as an extra regularizer. That work predicts the full distribution of annotator votes alongside the majority label. It was tested only on part-of-speech tagging with random splits, so whether it helps on held-out people is an open question we measure.

We train the concept encoder independently or sequentially, not jointly with the annotator heads. The bottleneck paper notes that joint training trades some concept fidelity for task accuracy, and any unconstrained route from text to label would let identity memorization leak around the concepts. The encoder is frozen before any annotator head is fit, so the only annotator-specific parameters are $w_a$ and $b_a$.

The main dataset is DEMO from Beyond Demographics. It has more than 147,000 annotations from 2,614 annotators with age, gender, race, and education, and we use its Offensiveness and Safety tasks. Running a separate fold for each of 2,614 annotators is too expensive, so we follow that paper's annotator split and generalize it to grouped cross-validation over annotators: ten folds, with no annotator appearing in both training and test. Within each held-out annotator, we sample $m \in \{0, 5, 10, 20, 50\}$ labels for calibration and evaluate on the rest. We repeat this over five random calibration draws.

The second dataset is IndoToxic2024: 43,692 Indonesian posts with ten demographic fields. There, male raters labeled 19.3% of posts toxic versus 13% for female raters. The third is the racist-language corpus of Reconsidering Annotator Disagreement: 291 annotators, overall Krippendorff's alpha of 0.39, and race-by-topic interactions. That corpus is small enough for true leave-one-annotator-out evaluation.

The primary metric is macro-F1 averaged per annotator on unseen annotators, plotted as a curve over $m$. The secondary metrics are three. The first is seen-annotator F1 on the instance split, which measures the accuracy cost of the bottleneck. The second is the false-positive rate on counterspeech. The third is the reliability of fitted weights, measured as the correlation between $w_a$ estimated from two disjoint halves of an annotator's labels.

Baselines

The naive reference point is the text-only population model. It is the same encoder with a single shared head, trained on majority or pooled labels and applied identically to every unseen annotator. Beyond Demographics reports that its best personalized model falls to this level on the annotator split. The file gives no numeric F1 for that split, so there is no reported result on Lacuna for DEMO annotator-split F1. We will reproduce this reference ourselves. Any method that cannot beat it at $m = 0$ is only matching the wall, which we expect. The real contest is at $m > 0$.

The strongest existing method is the fine-tuned LLM with annotator identity and demographic markers from the same paper (Llama 3 8B, "+ID+Attributes"). For unseen annotators it has no ID to use, so we give it a fair few-shot adaptation. It receives the same $m$ calibration labels, used either as in-context examples or for a low-rank adapter update. It gets the same number of tuning trials we spend on the prior $(\mu, \Sigma)$ and the same GPU budget per fold. Beating it means higher unseen-annotator macro-F1 at $m \le 20$, significant across folds and calibration draws, while staying within a few points of it on seen annotators.

The second strong competitor is a black-box personalization head. It uses the same frozen encoder, but each annotator's head reads the full pooled embedding rather than the concept vector, with the same Gaussian-prior posterior fitting. This is the ablation that isolates the bottleneck. A second ablation replaces the $k$ named concepts with a $k$-dimensional learned projection of the embedding, trained with the same annotator-prior objective. If the learned projection matches the named concepts, then the benefit comes from low dimension alone and the semantic concepts contribute only interpretability. We would report that honestly.

We also run the Fornaciari soft-label multi-task model as a population baseline, and demographic persona prompting as used in IndoToxic2024, where it raised LLM F1 by up to 0.47. For the counterspeech check, the reference is GPT-4. Gligoric et al. report a baseline false-positive rate of about 9% on hate-speech counterspeech, cut by 82.6% with chain-of-thought mitigation. Beating that means matching the mitigated rate through a concept edit, with no prompting.

Why not just…?

Why not just give the LLM a few of the new annotator's labels in context? This is the real threat, and it is the main head-to-head comparison above. Its weakness is that five to twenty examples give weak evidence about which feature drove each judgment. The concept model instead assigns the evidence to named axes through the prior. If few-shot LLM adaptation matches us at every $m$, our remaining advantage is only interpretability and intervenability. We would narrow the claim to that.

Why not just predict each person's label from their demographics? Beyond Demographics finds that demographic gains on seen annotators come from proxy identification of people with unique profiles and vanish on unseen annotators. IndoToxic2024 also finds that concatenating demographics hurts small BERT models. We still use demographics as an optional input to the prior mean $\mu$, and the $m = 0$ comparison shows whether they add anything beyond the text-only reference.

Why not just model the vote distribution with soft labels? Fornaciari et al. show soft labels help as a regularizer, but they predict the crowd, not a person. Their predictions are therefore identical for every unseen annotator, and at $m > 0$ they cannot improve. We include them to measure how much of the benefit comes from better population modeling rather than personalization.

Why not just personalize a black-box head on the embedding? At large $m$ this should match or beat us, because named concepts may be incomplete, which is a known weakness of bottlenecks. The learning-curve crossover point between the two heads is itself a result. It says how many labels per annotator one must collect before interpretability starts to cost accuracy.

Decisive Checks

Three results would settle the proposal. The first is a learning curve. The claim holds only if, on DEMO and IndoToxic2024, the concept-weight model beats both black-box personalization and few-shot LLM adaptation at $m \in \{5, 10, 20\}$ on unseen annotators, and also beats the learned-projection ablation of equal width. The second is stability of the weights. If $w_a$ fitted from disjoint halves of an annotator's labels correlate poorly, the "personal weights" are noise and the interpretive claim fails. On the racist-language corpus, we expect weights on identity attack to differ systematically by rater race in the direction the original authors found.

The third is intervention. On the counterspeech pairs, setting the use-versus-mention concept to "mention" should cut false positives across annotator heads without retraining. The test is whether the cut approaches the reduction Gligoric et al. achieved by prompting, while keeping the true-positive rate within their 2–3% margin. A failure of the first check with success on the other two would still give a useful auditing tool. A failure of the second check would mean the split between shared perception and personal judgment is wrong.