Tracing Residual Gender Bias Through Multi-Hop Attention Paths in BERT

Debiasing methods for word vectors are known to hide bias rather than remove it. After the "gender direction" is projected out, words that used to lean male still sit near each other, and so do words that used to lean female, and a simple classifier can recover the original bias from where a word sits. For BERT, a text encoder that builds each word's vector from its surrounding sentence, the strongest current fix works inside attention, the mechanism that decides which other words each word draws information from. It equalizes how much attention a neutral word like nurse pays to he versus she. Nobody has checked whether this leaves the same hidden clustering behind. There is a specific reason to suspect it does: gender information does not need a direct edge. Nurse can attend to caring, which already absorbed she one layer earlier.

This note proposes to treat BERT's lower-layer attention as a graph and measure bias as the multi-hop flow of attention from neutral words to demographic words. A random-walk importance score, of the kind used to rank words in a document graph, gives that flow, corrected for "hub" tokens that soak up attention everywhere. We would fine-tune BERT so this corrected flow is equal across counterfactual gender swaps. Before the intervention comes a diagnostic: how much of BERT's residual cluster bias is carried by attention routing at all, and how much is inherited from the input word vectors, which no attention method can touch. The first answer tells us whether attention-based debiasing is even the right lever. The second tells us whether one-hop equalization is enough.

Core Claim

Two claims are testable. First, in BERT, the Gonen–Goldberg style residual clusters survive one-hop attention equalization, because gender reaches neutral tokens through two- and three-hop attention paths in layers 0–3. Second, constraining degree-corrected multi-hop flow instead of direct attention weights disperses those clusters further. It should do so without the semantic cost seen when the embedding space is reshaped globally. If a path-length-one version of our objective does as well as the multi-hop version, the first claim is false and the proposal reduces to AttenD.

Where This Sits

The problem comes from Lipstick on a Pig. That paper showed that after hard-debiasing static vectors, K-means still recovers original gender groupings with 92.5% accuracy and an RBF-kernel classifier predicts original bias with 88.88% accuracy. It studied only static vectors.

MDR Cluster-Debias disperses those clusters by unfolding the curved embedding space. Classifier accuracy falls to 0.518, but IMDB accuracy drops from 80.9% to 68.5%. This constrains us: reshaping the whole representation space is too blunt, so any intervention must be local.

Debiasing Pretrained Text Encoders by Paying Attention to Paying Attention (AttenD) supplies both the motivation and the closest method. It found that attention bias in BERT concentrates in layers 0–3. It equalizes attention toward social-group tokens with teacher–student distillation, and it keeps SST-2 near 92.7%. It never measured whether residual clusters in the representations were dispersed, and its loss acts only on direct attention weights.

Multi-Feature Quantized Self-Attention for Fair Large Language Models (MQAR) instead inserts an adversarial quantization bottleneck in attention. It reports small downstream loss, but it models no paths.

Two further papers supply the graph machinery. Graphormer showed that attention shaped by graph structure outperforms plain attention on molecules. We borrow its idea that attention should be governed by paths rather than pairs, but we apply it as a soft penalty on BERT's own attention graph instead of an external molecular graph. Text Information Aggregation with Centrality Attention showed that recursive, eigenvector-style importance over token graphs can be backpropagated through. It used this only for pooling, not for measuring or constraining bias.

Residual2Vec supplies the correction. It showed that random walks over-weight high-degree nodes and removed this by comparing against a null random-graph expectation. In BERT the analogous hubs are [CLS], [SEP] and punctuation, which absorb attention regardless of meaning. An uncorrected flow penalty would mostly punish those tokens.

Research Program

We define attention flow through the bias-heavy layers. Let $A_l$ be the head-averaged attention matrix at layer $l$ for a sentence of $T$ tokens. Let $\bar A_l = \tfrac12 A_l + \tfrac12 I$ account for the residual connection. The flow from input positions into layer-3 representations is

$$F = \bar A_3 \bar A_2 \bar A_1 \bar A_0,$$

so $F_{t,g}$ is how much of token $t$'s layer-3 vector traces back to input token $g$ along all attention paths. To remove hub effects, we divide by the null expectation $n_g = \frac1T\sum_i F_{i,g}$, the flow an average token sends to $g$, giving $r_{t,g} = F_{t,g}/n_g$.

For each training sentence $s$ with a demographic word $g$, we form its counterfactual $s'$ with $g$ swapped to $g'$. We then penalize the gap in corrected flow for every neutral target token $t$:

$$\mathcal L_{\text{flow}} = \sum_{(s,s')}\sum_{t}\big|\log r_{t,g}(s) - \log r_{t,g'}(s')\big|,$$

added to the same distillation and negative-sampling terms AttenD uses for preserving semantics. Setting the product to a single layer's $A_l$ recovers a one-hop objective in the same codebase. This is the key ablation. Computing $F$ costs four $T\times T$ matrix products per sentence, so it avoids the roughly fivefold slowdown Gong et al. report for iterated power methods.

The diagnostic study comes first and is independently publishable. For roughly the 1,000 words most gender-skewed under BERT's own input-embedding projection, we sample many natural contexts per word. We then compute contextual versions of the Gonen–Goldberg tests at each layer: K-means recovery, RBF-SVM recovery, and nearest-neighbor bias correlation. Kolling et al.'s pointwise alignment measure (Investigating the Effects of Fairness Interventions Using Pointwise Representational Similarity) shows which individual words actually moved.

We decompose the recovered bias with two controls. In the first, gendered context words are replaced by neutral ones, which isolates routing. In the second, the input embeddings are projected before encoding, which isolates inheritance. We run this audit on vanilla BERT, on AttenD, and on our method.

Baselines

The naive reference is unmodified BERT-base. On CrowS-Pairs its bias score is 60.5% against an ideal of 50.0%, per CrowS-Pairs. Every other system must be read against that floor and against its own utility.

Post-hoc linear projection of contextual vectors is the classic fix. Lipstick on a Pig predicts it will leave clusters intact. We include it to confirm that the contextual audit reproduces the static-vector failure; if it does not, the audit itself is miscalibrated.

Counterfactual data augmentation means fine-tuning on gender-swapped text with the same sentence pairs our loss uses. It uses the same step count and the same hyperparameter-trial budget. Gender Bias in Coreference Resolution reports that swapping plus debiased vectors shrinks the WinoBias pro/anti gap from 27.2 to 2.1 F1.

The strongest competitor is AttenD. It reports up to a 9.53% StereoSet fairness improvement, HateXplain GMB-BNSP rising from 0.379 to 0.726, and SST-2 of 92.66% versus 92.78% for the original BERT. We would re-run it on identical data, distillation settings and trial budget rather than copying these numbers. The ablation isolates what we add: the one-hop version of our loss, which is essentially AttenD with degree correction.

MQAR is the representation-bottleneck alternative. No reported result was found on Lacuna for its individual StereoSet, CrowS-Pairs or WinoBias scores, so it is re-implemented under the same budget. The remaining ablations are no degree correction, flow over all twelve layers instead of 0–3, and a hard-mask variant that cuts high-flow demographic paths outright.

Beating AttenD means two things. The multi-hop method must match it on StereoSet stereotype score and CrowS-Pairs within seed variance. It must also cut contextual K-means and SVM bias recovery by more than the across-seed spread. Both must hold while SST-2 stays within half a point of the original. The StereoSet language-modeling score must also not fall disproportionately, since StereoSet reports that stereotype score tracks language-modeling ability with Spearman 0.87, so cheap fairness gains often come from damaged modeling.

Why not just…?

Why not just equalize direct attention, as AttenD does? This is the main threat. The one-hop ablation settles it directly. If one-hop and multi-hop flow disperse clusters equally, indirect routing is not where bias lives and our contribution collapses to the audit. The reason to expect a difference is mechanical: one-hop equalization leaves every two-step path through context words unconstrained. Nothing in AttenD's reported evaluation measures whether those paths still carry gender.

Why not just train an adversary to strip gender from the final representations, as MQAR does? This could plausibly disperse clusters as well as our method, and we treat it as a real competitor rather than a strawman. Our expectation is that an adversary suppresses whatever its own classifier can read, while neighborhood structure measured by K-means and nearest-neighbor correlation persists. That is the static-vector lesson of Lipstick on a Pig. If the audit shows the adversarial baseline also kills the neighbor-based measures, then path-level control is unnecessary, and we would report that.

Why not just fine-tune on gender-swapped data? Swapping changes the training distribution, but Dev et al. (summarized in the research file) found that bias woven into broader context survives it. Swapping also does nothing about the gender already stored in the input vector of nurse. The inheritance control in our audit quantifies exactly how much bias swapping can and cannot reach.

Why not just unfold the embedding manifold? MDR does disperse clusters, but at the cost of IMDB accuracy falling from 80.9% to 68.5%. A local, lower-layer constraint exists precisely to avoid that tax. We report IMDB and SST-2 for every system to show whether we avoid it.

Decisive Checks

The audit can falsify the whole direction. If the neutral-context control shows that nearly all residual cluster bias is inherited from input embeddings, then no attention-based method, ours or AttenD's, can remove it. The paper then becomes a negative result that redirects effort to the embedding layer.

Assuming routing matters, three comparisons decide the claims. The one-hop versus multi-hop comparison on contextual K-means and SVM recovery tests whether paths matter. The corrected versus uncorrected comparison tests whether hub tokens distort the penalty, predicting that the uncorrected loss mostly shifts attention on [CLS] and [SEP] with little fairness gain. The layers 0–3 versus all-layers comparison tests Gaci et al.'s localization as an intervention principle rather than only a diagnostic.

Extrinsic confirmation comes from the WinoBias pro/anti F1 gap with a BERT-encoder coreference model, HateXplain GMB-BNSP, and SEAT. Utility is checked on SST-2 and IMDB, all over at least five seeds.