Training on text pooled from many sources (news, forums, biographies, reviews) usually makes models more robust. It also quietly mixes together the different ways each source samples the world. If one source writes mostly about nursing and mostly about women, and another writes mostly about engineering and mostly about men, a model trained on the pool learns an occupation–gender link much stronger than the link inside any single source. The link comes from which source covers which topic, not from anything said within a source. This note proposes to undo that specific artifact. We discover latent topics with an embedding-based topic model, estimate how likely each document's source is given its topic (its "propensity"), and reweight training examples by the inverse of that probability. The reweighting simulates a pool in which every topic is drawn from every source in the same proportions.
The two-sentence version is this. Pooling domains can amplify social bias through between-domain correlations between topic and demographics, a kind of Simpson's-paradox aggregation. Inverse propensity weighting with domain as the "treatment" and discovered topic as the confounder should remove the amplification that pooling adds, while keeping the volume benefit of pooling. The approach only works if the topics are defined so they do not simply recover the source. That is the technical crux, and the note tests it directly.
Core Claim
Our hypothesis is that a measurable share of the bias in pooled-domain classifiers is pooling-induced: it exceeds the within-domain association averaged over domains. We further hypothesize that this excess can be removed by weighting each example by
$$ w_i = \mathrm{clip}\!\left(\frac{\hat P(d_i)}{\hat P(d_i \mid t_i)},\; c\right), $$
where $d_i$ is the source domain of example $i$, $t_i$ is its discovered topic, $\hat P(d_i)$ is the overall share of that domain, $\hat P(d_i \mid t_i)$ is the share of that domain within the topic, and $c$ is a clipping constant that controls variance. Under these weights, domain and topic are independent in the reweighted pool. This is the standard inverse-propensity construction of Recommendations as Treatments, which weights observed ratings by the inverse probability of being exposed. Here the "exposure" is which corpus a topic was collected from.
The claim that goes beyond prior work is this. Off-the-shelf embedding topic models make the weights useless, because contextual embeddings encode platform style. Clusters then split by source, a known failure of BERTopic-style pipelines on heterogeneous corpora (see Experiments on Generalizability of BERTopic on Multi-Domain Short Text). With source-pure clusters, $\hat P(d \mid t)$ approaches 0 or 1, overlap fails, and the weights explode. We propose to cluster on domain-purged embeddings: document embeddings from which a linear domain probe's directions have been projected out before dimensionality reduction (UMAP, a method that compresses vectors while keeping neighbors close) and density clustering (HDBSCAN, which groups dense regions and labels sparse points as noise). The prediction is concrete. Raw topics give high-variance weights with little bias reduction. Purged topics give overlapping propensities and a real reduction in pooling-induced bias.
Relation to Prior Work
Three lines of work motivate the proposal.
The first is about bias amplification. Bias in Bios shows that occupation classifiers' true-positive-rate (TPR) gender gaps correlate at 0.71–0.86 with occupational gender imbalance. It also shows that scrubbing names and pronouns leaves gender predictable at over 65% accuracy. The models amplify imbalance, and the imbalance runs through proxies, so it cannot be removed by deleting words. Feature-Wise Bias Amplification explains one route: gradient descent overweights many weakly predictive features. Topic vocabulary concentrated in particular sources is exactly such a pool of weak features.
The second line covers multi-domain pooling. WILDS and On the Impact of Cross-Domain Data on German Language Models establish that pooling and plain empirical risk minimization are hard to beat on average accuracy. Neither asks whether pooling changes the strength of demographic associations. WILDS also shows that Group DRO, a method that minimizes the loss of the worst-off group, helps on CivilComments only with oracle identity labels.
The third line is causal deconfounding. Deconfounded Recommendation treats a user's historical category distribution as a confounder and applies backdoor adjustment to stop recommenders from amplifying category skew. It uses discrete, given item categories inside factorization machines. We borrow its causal reading but replace the given categories with topics discovered from text, and we work on pooled text classifiers. Cross-Dataset Propensity Estimation estimates propensities from one log to debias another. Our setting is the text analogue, with corpora in place of logs.
Two papers constrain the design and also supply its evaluation. Men Also Do Laundry shows that signed single-attribute metrics hide amplification: 0.7 versus 11.9 on imSitu. It also shows that single-attribute fixes can backfire, with Data Repair raising multi-attribute amplification on COCO from 14.5 to 17.2. Our reweighting changes the joint distribution of topic, domain, and label, so we must report its absolute-difference multi-attribute metric and not only the gender TPR gap. Directional Bias Amplification shows that threshold calibration alone cut measured COCO amplification from 0.040 to 0.001. Any gain we report must survive per-group threshold recalibration, otherwise we have only fixed calibration. Its attribute-to-task versus task-to-attribute split also lets us check that reweighting does not push the model toward inferring gender from topic context.
Methods we do not adopt include What's in a Name?, which penalizes covariance between predictions and name embeddings, and MixPro, which mixes source and few-shot target embeddings. Both target a known proxy or a known target set. Ours targets a different cause, the between-domain sampling structure, and needs only domain-of-origin labels, which pooled corpora always have.
Research Program
The first step is diagnostic. We decompose the pooled model's bias into a within-domain part and a pooling-induced remainder. We train per-domain models and a pooled model on matched data volume, and measure each one's TPR gap and multi-attribute amplification on a common test set. If the pooled gap is not larger than the domain-weighted average of per-domain gaps, there is nothing for the method to remove, and we report that as a result.
The second step builds the topic layer. We embed documents with a sentence encoder and fit a linear probe for domain. We then project out the probe's top directions, using iterated null-space projection until probe accuracy nears chance, and run UMAP and HDBSCAN on the purged vectors. To label topics we use class-based TF-IDF, which ranks words by how distinctive they are of a cluster. HDBSCAN noise points are assigned to their nearest cluster for weighting. We measure overlap by the fraction of topics whose minimum $\hat P(d \mid t)$ exceeds 0.05, and by the effective sample size of the weights.
The third step trains the downstream classifier with the weighted loss, using stabilized and clipped weights. A doubly-robust variant, which adds an outcome model per topic and domain, is a stretch goal if variance proves limiting. The key unmeasured-confounding risk is that something other than topic drives both domain membership and demographic skew. We probe it by sensitivity analysis: we vary purging depth and topic granularity and check whether conclusions flip.
Baselines
The naive reference is unweighted pooled empirical risk minimization on the same backbone and data. On CivilComments-WILDS the comparable reported figure is ERM worst-group accuracy of 56.0% (WILDS).
The strongest existing competitor on worst-group text robustness is Group DRO with oracle identity groups, which reaches 70.0% worst-group accuracy on CivilComments-WILDS in the same paper. It uses labels our method does not use. We therefore treat matching it within a few points, without identity labels, as success there, and do not require beating it.
On Bias in Bios, the strongest label-free debiasing competitor is CoCL from What's in a Name?. The file reports CoCL's Adult result: RMS TPR gap 0.299 down to 0.163 at a 0.5% balanced-accuracy cost. No reported Bios number was found on Lacuna, so we rerun CoCL ourselves. Beating it means a lower RMS TPR gap at equal or better accuracy, together with no increase in multi-attribute amplification.
We also run per-group threshold calibration following Directional Bias Amplification, both as a baseline and applied on top of every method.
The ablations isolate the key ingredient. Weighting by domain alone (domain-balanced sampling) tests whether topics matter at all. Weighting by raw, unpurged BERTopic topics tests whether purging matters. Random topic assignments with matched cluster sizes check that any gain does not come from the variance-reduction side effects of reweighting.
All methods share the encoder, the training data, and an equal hyperparameter search budget, for example the same number of trials over learning rate and the method's own knob (clip constant, DRO step size, CoCL weight). Model selection uses a validation set that carries no demographic labels, except for the oracle Group DRO row.
Why not just…?
Why not just balance the domains? Equalizing how much each source contributes leaves intact the correlation between which source covers which topic, and that correlation is what we claim drives pooling-induced bias. In the semi-synthetic experiment below, domain balancing should leave the pooled-minus-within gap largely unchanged. If it removes most of the gap, our topic layer is unnecessary and we will say so.
Why not just run Group DRO on discovered clusters instead of reweighting? This is a real threat. Clusters can serve as pseudo-groups, and worst-group optimization does not need propensities. We run it with the identical purged clusters and with topic-by-domain cells. Our expected advantage is that Group DRO targets worst-group loss and not the association structure. Its tendency to upweight whole cells could therefore backfire on multi-attribute metrics the way single-attribute fixes did in Men Also Do Laundry. If cluster Group DRO matches our method on both TPR gap and multi-attribute amplification, the contribution reduces to the purged-topic construction, and we will frame it that way.
Why not just recalibrate thresholds afterward? Calibration explained almost all measured amplification on COCO in Directional Bias Amplification. For that reason we report every method after recalibration. Calibration cannot change which proxies the representation uses. We therefore also measure gender recoverability from scrubbed-text representations, where Bias in Bios found over 65%. Reweighting should lower it and calibration cannot.
Why not just scrub identity words? Bias in Bios already shows that proxies survive scrubbing. We include scrubbing only as a cheap row, expected to be weak.
Proposed Experiments
The decisive experiment is semi-synthetic, because it gives ground truth. We split Bias in Bios (397,340 biographies, 28 occupations) into four synthetic "domains." Each domain has a controlled occupation mix and a controlled gender ratio, and each carries a distinct register, applied by deterministic style transforms such as casing, contraction, and templated forum or news framing, so that domain is detectable from style as real platforms are. We set the within-domain gender–occupation association constant across domains. Any excess association in the pooled model is then pooling-induced by construction. We sweep the strength of the domain–topic correlation.
The prediction is that the pooled TPR gap grows with that correlation, that our method holds the gap near the within-domain level across the sweep, and that the raw-topic ablation fails at high correlation through weight explosion. A falsifying outcome would be that raw-topic weighting performs as well as purged-topic weighting, or that domain balancing suffices.
The naturalistic experiment uses CivilComments-WILDS with identity metadata held out. We test whether purged-topic weighting, using only the comment's coarse source labels, raises worst-group accuracy from the 56.0% ERM reference toward the 70.0% oracle Group DRO figure, with Amazon-WILDS as a second pooled-source check for accuracy cost.
Across all settings we report accuracy, RMS TPR gap, absolute-difference multi-attribute amplification over {occupation, gender, domain, topic}, both directional amplification terms, effective sample size, and propensity overlap. Every number is shown before and after threshold recalibration, with confidence intervals over seeds. The confidence intervals address the observation in Directional Bias Amplification that equally accurate models can differ in fairness.
References
- Recommendations as Treatments: Debiasing Learning and Evaluation
- Experiments on Generalizability of BERTopic on Multi-Domain Short Text
- Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting.
- Feature-Wise Bias Amplification
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- On the Impact of Cross-Domain Data on German Language Models
- Deconfounded Recommendation for Alleviating Bias Amplification
- Cross-Dataset Propensity Estimation for Debiasing Recommender Systems
- Men Also Do Laundry: Multi-Attribute Bias Amplification
- Directional Bias Amplification
- What's in a Name? Reducing Bias in Bios without Access to Protected Attributes
- Few-shot Adaptation to Distribution Shifts By Mixing Source and Target Embeddings