Where Bias Enters a Label Description: Separating Pretrained and Historical Channels in Description-Based Classifiers

A growing family of text classifiers no longer predicts an arbitrary class index. It scores how well an input matches a written description of each class. For example, a model asks whether a biography entails "This person works as a nurse." This is attractive because the description carries knowledge the language model already has, which helps with rare and unseen classes. It also means the description arrives loaded with whatever the model absorbed during pretraining about who nurses, surgeons, and software engineers usually are. Fairness work on occupation classification has studied two sources of bias: the input text, which leaks gender through proxy words, and the historical labels, which encode past hiring patterns. It has never asked whether the label text itself is a third, independent source.

We propose to measure that third source and then remove it. The core idea is that a description-based classifier can be tested before it ever sees a historical label. A zero-shot entailment model scoring biographies against occupation descriptions has learned nothing from past hiring decisions. Any gender gap it shows must come from pretrained associations triggered by the description and the input. Comparing this zero-shot gap with the gap after fine-tuning on historical labels separates the pretrained channel from the historical one, and lets us ask whether training on biased labels amplifies an association the description already carried. We then select class descriptions, from a pool of candidates generated by a language model, so that each class's score barely changes when the gender of the biography is counterfactually swapped. Accuracy is constrained to stay at the level of the original descriptions.

Core Claim

In cross-encoder classifiers, where the description and the input are read jointly by one network as in natural language inference (NLI, deciding whether one text entails another), the wording of the class description is a causal lever on the gender true-positive-rate gap. Holding the model and inputs fixed and changing only the description should move the gap measurably. Descriptions chosen to be invariant under gender swaps should then reduce the gap more than input scrubbing or input-side erasure at matched accuracy, and the reduction should persist after fine-tuning on historical labels.

We state the claim's boundary up front. For a dual encoder, the input and description are embedded separately and scored by an inner product, so erasing a gender direction from the input with an orthogonal projection is mathematically the same as erasing it from the description. Label-side intervention adds nothing there. The claim is specific to models where the two texts interact nonlinearly, which includes the NLI formulation used for multi-label value classification by Cepeda et al. at SemEval-2023 Task 4. We will verify the dual-encoder equivalence empirically as a sanity check on our measurement pipeline.

Grounding in Prior Work

The motivation comes from Bias in Bios. It showed that classifiers trained on 397,340 biographies have per-occupation true-positive-rate gender gaps that correlate with each occupation's existing gender imbalance at 0.71 to 0.86. It also showed that scrubbing names and pronouns leaves gender recoverable with over 65% accuracy. Those classifiers used discrete output heads, so the label side never had a chance to contribute.

What's in a Name? uses continuous embeddings of personal names as stand-ins for demographics and penalizes covariance between predictions and those embeddings. It embeds the input entity, not the target class, which is exactly the side we change.

Feature and Label Embedding Spaces Matter is the closest prior work. It recognizes that bias lives in the label projection as well as in the features, and trains a separate head per protected group. Its label embeddings are learned weight vectors rather than readable text, it requires discrete group labels at training time, and it is evaluated only on images. Our label side is natural language. Our intervention edits words rather than weights, and it needs group information only in the form of gender-swapped counterfactuals.

Two papers constrain the design. SPLINCE shows that plain linear erasure such as LEACE destroys task information when gender and occupation are correlated, and fixes this by preserving covariance with the label. When the label is historically biased, as Jiang and Nachum argue, preserving covariance with that label also preserves the bias it carries. Our zero-shot measurement avoids the observed label entirely. On Group Sufficiency Under Label Bias proves that enforcing fairness constraints against biased labels produces miscalibration relative to the unobserved true labels. For this reason we report zero-shot behavior as a label-free reference rather than treating fine-tuned fairness metrics as ground truth.

The method for building candidate descriptions borrows from StructCBM, which uses a language model in a generate–predict–refine loop to write prototypical and contrastive concepts for classes. We keep the generation step. We replace its error-driven refinement with a selection criterion based on counterfactual invariance. Fair confident learning gives a warning here: error-driven feedback treats minority-group errors as concept deficiencies. A StructCBM-style loop run on Bios would likely write gendered concepts. We test this directly.

Research Program

The first component is a decomposition study on Bias in Bios. Every system is a cross-encoder NLI model scoring each biography against one description per occupation. There are three training regimes. In the zero-shot regime, a pretrained NLI model is used with no Bios labels. In the few-shot regime, the model is fine-tuned on 10 examples per occupation. In the full regime, it is fine-tuned on all training labels. For each regime we report three things: the per-occupation true-positive-rate gap, its root-mean-square across occupations, and the correlation of the per-occupation gap with each occupation's female share, the statistic De-Arteaga et al. reported.

The key manipulation varies only the description while holding everything else fixed. We use roughly twenty candidate wordings per occupation, ranging from job titles to task descriptions to wordings an LLM generates in StructCBM style. If the true-positive-rate gap varies substantially across wordings that have similar accuracy, the label channel is real. The amplification question asks whether the full-regime gap for an occupation is predicted by its zero-shot gap beyond what female share alone predicts.

The second component is the intervention, called counterfactual-invariant description selection. Let $s(x, d)$ be the model's entailment probability for biography $x$ and description $d$. Let $\tilde{x}$ be $x$ with gendered words swapped and names replaced. For each occupation $c$, among candidates $D_c$, we choose

$$ d_c^* = \arg\min_{d \in D_c} \; \mathbb{E}_{x}\big|s(x,d) - s(\tilde{x},d)\big| \quad \text{subject to} \quad \mathrm{Acc}_c(d) \ge \mathrm{Acc}_c(d_c^{0}) - \epsilon, $$

Here $d_c^{0}$ is the plain job-title description, $\mathrm{Acc}_c$ is validation recall for class $c$, and $\epsilon$ is a fixed tolerance chosen on validation data. The selection uses only counterfactual pairs, never gender labels. It runs once before fine-tuning, and the selected descriptions are then held fixed through fine-tuning.

The swap sensitivity measures how much the description makes the score depend on gender cues, and it can be computed without labels. Selection is cheap and auditable: the output is a set of readable sentences one can inspect. We will also report which kinds of wording tend to be selected, for example task verbs versus status nouns.

The third component is a longitudinal check. We reuse the compounding simulation from Bias in Bios to ask whether selected descriptions slow the drift De-Arteaga et al. describe, in which a field starting at 10% women dwindles toward 0%.

Baselines

The naive reference is a RoBERTa-style classifier with a discrete 28-way output head, fine-tuned on Bios with no fairness intervention. This is the setting De-Arteaga et al. studied, and it anchors the familiar correlation range of 0.71 to 0.86. We also include the same NLI model with plain job-title descriptions and no selection. This isolates what moving to descriptions does by itself.

Input scrubbing, removing names and pronouns, is applied to both the discrete and the description-based models. It is the cheapest intervention and the one practitioners reach for first. Every comparison in this note is reported both with and without scrubbing, because the label channel should matter most once the input channel is partly closed.

The strongest existing competitor is input-side erasure with SPLINCE. It was evaluated on Bias in Bios and keeps task accuracy where LEACE does not. We apply it to the cross-encoder's pooled representation before the classification layer and use the same backbone, data split, and tuning budget. No reported result was found on Lacuna for SPLINCE's numerical accuracy or true-positive-rate gap on Bias in Bios; the paper reports only qualitative robustness relative to LEACE and SAL. We will therefore rerun it ourselves.

The second competitor is CoCL, which uses name embeddings as a proxy for protected attributes. On Bios it reports only "significantly lower" bias, with no numbers on Lacuna. On Adult it reports a root-mean-square true-positive-rate gap falling from 0.299 to 0.163 at a 0.5% balanced-accuracy cost. We will recompute it on Bios.

Beating a competitor means a lower root-mean-square gap and a lower correlation with female share at equal or better overall accuracy, within the $\epsilon$ tolerance, averaged over five seeds. Description selection should also compose with SPLINCE to give a further reduction, which would show it reaches a channel SPLINCE does not.

Three ablations isolate the key ingredient. In the first, a description is chosen at random from the same candidate pool, which controls for simply having different wording. In the second, the description is chosen by accuracy alone, which controls for selection pressure without the invariance criterion. In the third, a StructCBM-style error-driven refinement loop is used, which tests the prediction from fair confident learning that error feedback writes gendered concepts. All variants use the same candidate pool and the same number of LLM calls.

Why not just…?

Why not just scrub the input? Bias in Bios already shows gender remains recoverable at over 65% accuracy after scrubbing, and that the gap–imbalance correlation stays high. Scrubbing also does nothing to a description such as "a nurse who cares for patients." That description can still pair with residual proxies like "women's health" in the biography. Our scrubbed-input runs measure how much gap remains for the label side to explain.

Why not just erase gender from the input representation with SPLINCE or LEACE? For dual encoders, this would make our method redundant, and we say so and verify it. For cross-encoders, the description and biography interact across all layers, so erasing gender at the final pooled layer leaves the earlier interaction in place. The real threat is that final-layer SPLINCE captures most of the gap anyway. The decisive test is whether description selection adds a reduction on top of SPLINCE. If it does not, the label channel is real but redundant with output-side erasure, and we will report that.

Why not just hand-write neutral descriptions? This is the most serious alternative, because it is even cheaper than our selection. We include hand-neutralized descriptions, with gendered and status-laden terms removed by annotators, as a baseline. If they match the selected descriptions, the contribution shrinks to the diagnosis plus a simple guideline. That would still be useful but much smaller. We expect selection to win only if the harmful associations are not obvious to people, for example in task verbs that correlate with gender. The swap-sensitivity scores of the hand-written descriptions will show whether that is the case.

Why not just fine-tune on gender-swapped data, or adjust thresholds per group afterwards? Counterfactual data augmentation is run with the same swap function and training budget. It targets the input side during training, and it is unclear whether it undoes associations triggered by the description. Per-group thresholds need gender labels at deployment. Jiang and Nachum found post-processing costlier in accuracy than modeling the bias directly, and the group-sufficiency analysis shows it miscalibrates against the true labels when the historical labels are biased. We report thresholding only as a reference point that uses attributes.

Decisive Checks

The project stands or falls on three results on Bias in Bios. First, in the zero-shot regime, the correlation between the per-occupation gap and female share must be clearly positive. That would show pretrained associations, absent any historical labels, already produce part of the pattern De-Arteaga et al. attributed to training. Second, at fixed model and inputs, the gap must vary across descriptions of matched accuracy by more than the variation across random seeds. Third, after full fine-tuning, invariance-selected descriptions must beat plain titles, hand-neutralized descriptions, and the random-selection and accuracy-only ablations on root-mean-square gap at matched accuracy, and must add to SPLINCE when combined with it.

A null result on the first check together with a positive result on the third would mean descriptions mainly modulate how historical bias is learned rather than importing new bias. That is still a publishable distinction. Null results on both the second and third checks would mean the label channel does not matter in practice, which would itself be a useful finding for practitioners adopting description-based classifiers.