Bias amplification is the finding that a trained model exaggerates skews in its training data. The standard example: if "cooking" images are 66% women in training, the model predicts women 84% of the time. Recent work argues that looking at one attribute at a time hides much of this, and that we must audit combinations of attributes such as "unloading" together with "indoors". Most such combinations are rare. With a few dozen examples, an estimated gender fraction can easily be off by ten points from pure chance. The newer metrics also average absolute differences so that positive and negative gaps do not cancel. Taking the absolute value of a noisy number always gives a positive average. So these metrics report amplification even for a model that reproduces its training distribution exactly, and the spurious amount grows as combinations get rarer. This proposal builds a noise-corrected estimator and a matching statistical test for multi-attribute bias amplification. It then uses them to ask which published amplification numbers, and which published "mitigation backfires", survive once noise is removed.
The correction comes from Poissonization, a standard trick in distribution testing. Instead of fixing the number of samples at $n$, one draws the sample size itself from a Poisson distribution with mean $n$. The count in each category then becomes an independent Poisson variable, rather than part of a coupled multinomial vector whose counts must sum to $n$. Under that independence, simple count corrections give exactly unbiased estimates of squared differences between distributions, and their variances add up category by category. This is the machinery behind optimal closeness testers. We carry it over to per-combination demographic fractions. The result is an amplification metric with zero expected value under no amplification, calibrated confidence intervals, and a test that needs no retraining or permutation.
Core Claim
The closest bias-amplification work does not treat its metrics as finite-sample estimators. Men Also Do Laundry introduces multi-attribute metrics (MultiMALS and a directional variant). It reports that switching from raw to absolute differences moves measured imSitu amplification from 0.7 to 11.9. It also reports that Data Repair, a method that resamples training data to reduce single-attribute bias, raises multi-attribute amplification on COCO from 14.5 to 17.2. Both effects are exactly what a noise floor would produce. Absolute values inflate noise into signal. Data Repair resamples and thins the data, which shrinks per-combination counts and raises that floor. Our claim is that a sizable, measurable part of these numbers is sampling bias. A Poissonized unbiased estimator of squared amplification separates the real part from the noise.
This is distinct from the Poissonization papers in the file. Optimal Algorithms for Testing Closeness of Discrete Distributions builds the corrected statistic $(X_i-Y_i)^2 - X_i - Y_i$ for comparing two abstract distributions. Improved Distribution Estimation in ℓ∞ separates error from frequent items and rare items for plain frequency estimation. Neither addresses conditional demographic fractions, many-combination audits, or fairness claims. The novelty lies in the target, not the trick. The deliverable is an audit tool, and the scientific question is how much of the multi-attribute literature's evidence holds up.
Research Program
Write $m$ for an attribute combination, such as a verb with a location, or a set of co-occurring objects. For each $m$, let $\pi^D_m$ be the fraction of training instances with protected value $a$ (for example, women) among those containing $m$. Let $\pi^M_m$ be the same fraction among the model's test-time predictions. The target is the weighted squared amplification
$$\mathcal{A} = \sum_m w_m \left(\pi^M_m - \pi^D_m\right)^2,$$
where $w_m$ is the combination's share of occurrences. We report $\sqrt{\mathcal{A}}$ on the same scale as existing metrics.
Under Poissonization, the training and prediction counts for different combinations are independent. Condition on a combination's total count $T_m$ and its count $X_m$ with value $a$. Then $X_m(X_m-1)/(T_m(T_m-1))$ is an unbiased estimate of $\pi_m^2$, and the cross term $\hat\pi^M_m\hat\pi^D_m$ is unbiased because training labels and test predictions come from disjoint samples. Summing these per-combination pieces gives an estimator $\hat{\mathcal{A}}$ with expectation exactly $\mathcal{A}$. Its variance is a sum of per-combination terms, and each term can itself be estimated without bias. This gives confidence intervals and a one-sided test of whether $\mathcal{A}$ exceeds a threshold $\epsilon^2$.
Combinations seen only once contribute nothing to the square terms, but they are not discarded. Combinations seen twice contribute little. So the estimator quietly down-weights exactly the cells where the absolute-value plug-in is most inflated, without an arbitrary cutoff. The same pieces also give a directional version. Replacing the model's predicted attribute with a predictor trained only on task outputs follows the task-to-attribute split of Directional Bias Amplification.
Poissonization is exact only when the sample size is itself Poisson. Real datasets have fixed size, so we handle this two ways. First, we implement an exact Poissonized audit by resampling each split with a Poisson-distributed size. Second, we check on synthetic data that the fixed-size estimator matches it; this is the depoissonization concern the Poissonization literature flags. We also derive the expected value of the existing absolute-difference metric under zero amplification. It scales roughly as $\sum_m w_m\sqrt{\pi_m(1-\pi_m)/T_m}$, because the mean absolute deviation of a binomial fraction behaves like its standard deviation. This gives each published number a predicted noise floor that can be checked before any new experiment.
This does not dispute that real amplification exists. An Effective Theory of Bias Amplification proves amplification arises even in balanced data from differences in feature covariance and label noise. That is genuine signal, and our estimator should keep it. The claim concerns rare-cell audits specifically. Single-attribute results on frequent verbs, such as those behind Men Also Like Shopping, should be largely unaffected, and that is itself a prediction.
Baselines
The naive reference point is the published plug-in multi-attribute metric from Men Also Do Laundry, computed with its code on the same imSitu, COCO and CelebA splits and models. It is the strongest existing method because it is the only multi-attribute amplification measure in the file. It reports 11.9 on imSitu with absolute differences, and 14.5 rising to 17.2 on COCO under Data Repair. We regard it as beaten if two things hold on data with known ground truth. First, our estimator's bias stays near zero while the plug-in drifts with the number and rarity of combinations. Second, our intervals achieve nominal coverage.
A minimum-support filter is the obvious practitioner fix: drop combinations seen fewer than $k$ times. We sweep $k$ over the same grid we would give any tuned method, and pick the best $k$ on held-out synthetic settings. It beats nothing if it lowers bias only by discarding the rare combinations where multi-attribute bias lives, which we measure as lost recall of planted amplified combinations.
Bootstrap bias correction and a permutation null are the strongest general-purpose competitors. We run both with 1,000 resamples, which is far more compute than our closed-form estimator uses. For single-attribute calibration, we also report the threshold-tuning result of Directional Bias Amplification, which reduced COCO amplification from 0.040 to 0.001. This separates calibration error from sampling error.
Two ablations isolate the key ingredient. One keeps the unbiased per-cell correction but computes variances as if counts were multinomially coupled, ignoring Poisson independence. The other keeps Poisson variances but uses the uncorrected squared plug-in. No reported result was found on Lacuna for any bias-corrected or interval-equipped amplification metric.
Why not just…?
Why not just bootstrap the existing metric? Bootstrapping gives a spread, but the interval is centered on a biased quantity. First-order bootstrap bias correction for absolute values of near-zero quantities is known to be unstable, because the bias depends on the unknown true gap. This is a genuine competitor, not a strawman. The synthetic experiments compare bias, root-mean-squared error and coverage head to head. If bias-corrected bootstrap matches us, the contribution shrinks to a cheaper closed-form equivalent plus the audit findings, and we will say so.
Why not just drop rare combinations? Rare combinations are exactly where Men Also Do Laundry located surprises such as {unloading, indoors} skewing female. Filtering trades noise for blindness. The planted-combination recall experiment measures that trade directly.
Why not just smooth the fractions toward the overall rate? Additive smoothing shrinks every gap, the real ones included, so its bias depends on the amplification it is trying to measure. Our estimator is unbiased at every true value. We include smoothing with tuned strength in the synthetic comparison.
Proposed Experiments
The first experiment is synthetic. Using the attribute and gender marginals of imSitu and COCO, we simulate a "model" whose predictions are fresh draws from the training conditionals, so true amplification is exactly zero. We also plant known amplification in a chosen set of rare and common combinations. We vary test-set size and combination order ($|m|=1,2,3$). The decisive check is that the plug-in metric reports positive amplification that grows with order and falls with sample size, roughly following the derived floor, while $\hat{\mathcal{A}}$ stays centered on zero with nominal interval coverage.
The second experiment is a re-audit of published results. We retrain the reported imSitu and COCO models, including the Data Repair and corpus-constrained variants, and compute both metrics. The central falsifiable question is whether COCO's 14.5 to 17.2 backfire keeps a confidence interval excluding zero under $\hat{\mathcal{A}}$. We also measure how much of imSitu's 11.9 exceeds its predicted noise floor. Single-attribute results from Mitigating Gender Bias Amplification in Distribution by Posterior Regularization, which reports average amplification cut from 3.2% to near zero, serve as a control that should change little.
A third, smaller experiment replicates on CelebA. The multi-attribute paper reports higher amplification there but no number, so we check whether that ordering survives correction. Either outcome is informative. If backfires survive, the field gains a calibrated way to confirm them. If they vanish, several mitigation comparisons need to be redone with intervals.
References
- Men Also Do Laundry: Multi-Attribute Bias Amplification
- Optimal Algorithms for Testing Closeness of Discrete Distributions
- Improved Distribution Estimation in $\ell_\infty$
- Directional Bias Amplification
- An Effective Theory of Bias Amplification
- Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
- Mitigating Gender Bias Amplification in Distribution by Posterior Regularization