Two-Game Shapley Gating for Adversarial Domain Adaptation

Adversarial domain adaptation trains a feature extractor to fool a domain discriminator, a small network that tries to tell whether a feature vector came from the labeled source data or the unlabeled target data. The idea is that features the discriminator cannot separate will also transfer well. The well-known failure is that this pressure erases more than domain style. It also flattens the feature directions that separate similar classes. This proposal treats each channel of the extractor's final feature vector as a player in two cooperative games. In the first game the payoff is how well the label classifier predicts. In the second it is how well the discriminator identifies the domain. We estimate each channel's Shapley value in both games during training: its average marginal contribution over all subsets of channels. We then use the resulting two-dimensional credit map to decide, channel by channel, how much of the reversed adversarial gradient each channel receives.

Both games are played only through the two small heads that sit on a 256-dimensional bottleneck, so evaluating a subset of channels costs a forward pass through a tiny network rather than through the backbone. That is what makes a principled Shapley estimate affordable inside every training step. The proposal also asks a question that the literature currently answers in two contradictory ways. Should channels that carry both class and domain information be aligned harder, or protected from alignment?

Core Claim

The claim is that per-channel Shapley credit, estimated with a sampler that updates every player from each subset evaluation, gives a better signal for routing adversarial pressure than first-order gradient saliency. The reason is specific to the domain game. Once the discriminator becomes confident, its sigmoid output saturates and its gradients with respect to individual channels vanish. In addition, domain cues such as lighting and texture are spread redundantly across many channels, so removing any single channel changes little even though removing a group changes a lot. Shapley values split credit fairly across redundant players and use finite subset differences rather than local slopes. They should therefore keep tracking domain information exactly when gradient saliency goes blind.

For the classifier game the argument is weaker, and we say so plainly. With a linear classification layer and a mean-feature baseline, the Shapley value of a logit equals gradient times input. Any advantage on that side can only come from using a nonlinear payoff, the log-likelihood of the true label. This asymmetry is itself a testable prediction: Shapley should beat saliency mainly through the domain game.

How It Relates to Prior Work

The method comes from the efficient-Shapley literature. Approximating the Shapley Value without Marginal Contributions (SVARM) writes each Shapley value as a weighted average of payoffs of subsets containing the player minus a weighted average of payoffs of subsets excluding it. One subset evaluation therefore updates the estimate for every channel at once, with variance growing only logarithmically in the number of players for the stratified variant. Its $O(n^2)$ memory for per-stratum statistics is the known limitation. With a 256-channel bottleneck this is about 65k entries, which is trivial. We reuse SVARM unchanged. As a second estimator we test DASP, which propagates means and variances of random subsets analytically through a network. Its Gaussian assumption is most defensible on a shallow multilayer perceptron like a discriminator head, which is exactly where we apply it. Stratified Empirical Bernstein sampling supplies the variance-adaptive allocation across subset sizes, which we use to decide how many subsets to draw per size.

The problem framing comes from work showing that adversarial alignment damages class structure. Batch Spectral Penalization (BSP) shows that adversarial training lets the top singular value of the feature matrix dominate, and it penalizes the top singular values uniformly. Matrix Rank Embedding (MRE) imposes low rank within classes and high rank across classes. Both act on the whole batch geometry and cannot say which channels carry style and which carry class identity. That per-channel distinction is the thing we add.

The closest mechanism, and a partial contradiction, is BIWAA. It weights features entering the discriminator by classifier gradient saliency, so that alignment concentrates on task-relevant foreground. That gain alone is worth 9.7 points over DANN on Office-Home. BIWAA's policy is to align class-relevant channels harder, which is the opposite of the BSP and MRE intuition that alignment should keep away from discriminative directions. We do not assume either is right. The two-game map separates channels into four kinds: style-only, class-only, entangled, and inert. Both policies can then be run on the entangled channels under identical conditions.

Two papers already connect Shapley values and domain adaptation, and neither does what we propose. FDAT routes tabular features by per-feature one-dimensional Wasserstein shift and applies SHAP only after training, for explanation. Shapley never touches training there. SVFair uses Shapley values over task gradients to weight cooperative multi-task losses. It values loss heads, not representation channels, and it does not address the deliberately antagonistic gradient introduced by gradient reversal.

Research Program

The base system is DANN and CDAN+E on a ResNet-50 backbone with a 256-channel bottleneck $z$. DANN adds a gradient reversal layer, which is the identity going forward and multiplies the gradient by $-\lambda$ going backward. CDAN conditions the discriminator on classifier predictions and weights samples by prediction entropy.

At each training step, and for a mini-batch, we define two payoffs over subsets $S$ of channels. Channels outside $S$ are replaced by the running mean of $z$ pooled over both domains. Pooling over both domains matters: a mean-replaced channel then carries no domain signal of its own, which addresses the known sensitivity of Shapley values to the choice of "missing" value. The class payoff $v_{\text{cls}}(S)$ is the mean log-probability of the true label on source samples. The domain payoff $v_{\text{dom}}(S)$ is the mean log-probability the discriminator assigns to the correct domain on source and target samples. SVARM draws $T$ subsets per step and evaluates both heads on all of them in one batched pass. The estimates $\phi^{\text{cls}}_j$ and $\phi^{\text{dom}}_j$ are smoothed across steps with an exponential moving average, because the discriminator is a moving target.

The gate replaces the scalar reversal coefficient with a per-channel one:

$$ \tilde g_j = -\lambda\, \alpha_j \, \frac{\partial \mathcal{L}_{\text{dom}}}{\partial z_j}, \qquad \alpha_j = \alpha_{\min} + (1-\alpha_{\min})\, \sigma\!\big(\kappa(\hat\phi^{\text{dom}}_j - \beta\,\hat\phi^{\text{cls}}_j)\big). $$

Here $\tilde g_j$ is the reversed gradient that reaches channel $j$, $\hat\phi$ are rank-normalized Shapley estimates, $\sigma$ is the logistic function, and $\kappa$ sets the gate's sharpness. The parameter $\beta$ sets how strongly class credit protects a channel. Setting $\beta>0$ gives the protect policy, and $\beta<0$ approximates BIWAA's align-what-matters policy on entangled channels. The floor $\alpha_{\min}$ keeps some alignment pressure on every channel, since the target-error bound behind adversarial adaptation still requires the overall domain gap to shrink.

The discriminator itself sees ungated features, so it remains an honest measure of residual domain information. Only the extractor's update is reshaped.

Three analyses accompany the method. The first checks the estimator: on frozen checkpoints, compare SVARM and DASP estimates at the training budget against a 100k-subset Monte Carlo reference, measuring rank correlation and error as $T$ varies. The second tracks the representation: the singular-value spectrum of source and target features over training, in the style of BSP's $\sigma_1$-dominance diagnostic, together with per-class target accuracy on the rarest classes. The third measures cost: wall-clock overhead per step relative to CDAN+E, at $T \in \{64, 256, 1024\}$ and Shapley refresh intervals of 1, 5, and 20 steps.

Baselines

The naive reference points are source-only training and plain DANN. On MNIST→MNIST-M, DANN reports 52.25% for source-only and 76.66% adapted. We run both inside our codebase so that every later number is measured against the same implementation.

The base we build on is CDAN+E, which reports 87.7% average on Office-31 and 65.8% on Office-Home in Conditional Adversarial Domain Adaptation. It is the right base because its entropy weighting already handles sample-level noise, so any gain from our method must come from channel-level routing.

The strongest matched competitor is BSP+CDAN, which reports 88.5% on Office-31, 66.3% on Office-Home and 75.9% on VisDA-2017 in Batch Spectral Penalization. It targets the same failure we target, with a cheap uniform penalty. We also run MRE (Matrix Rank Embedding), whose file entry reports only relative gains, for example +8.5 points over CDAN on VisDA-2017; we reproduce it rather than quote an absolute number. The highest reported numbers belong to BIWAA, at 90.5% on Office-31, 71.5% on Office-Home and 79.44% on DomainNet (BIWAA). That system also includes iterative label-distribution alignment and centroid separation, which are orthogonal to our gate. For a fair comparison we therefore run BIWAA's feature weighting alone on CDAN+E, and separately add our gate to the full BIWAA pipeline.

Every method uses the same backbone, bottleneck, data splits, number of iterations, and three seeds. Each gets the same hyperparameter budget of 20 trials, selected with a reverse-validation criterion that uses no target labels.

The ablation that isolates the key ingredient keeps the gating rule fixed and changes only the attribution. One variant replaces $\hat\phi$ with gradient times activation on both heads. A second replaces only the domain-side attribution, testing the prediction that the advantage lives in the domain game. A third uses random permutations of the Shapley scores. Beating the strongest competitor means exceeding BSP+CDAN by at least one point of average accuracy on Office-Home and VisDA-2017 across seeds, at under 30% wall-clock overhead. The gradient-saliency ablation should land closer to CDAN+E than to the Shapley version.

Why not just…?

Why not just weight samples by prediction entropy? CDAN+E already does this, and our method runs on top of it. Entropy acts on which samples matter, not on which feature directions get erased, so it cannot stop the spectral collapse that BSP documents. BIWAA also notes that entropy is unreliable under label shift, because wrong predictions can be confident. If CDAN+E with a tuned threshold matched our gate, the per-channel story would be refuted. The CDAN+E row against the gated row settles this directly.

Why not just use backprop saliency, as BIWAA does? This is the most serious threat. For the classifier's logits, Shapley values and gradient times input coincide exactly under our baseline choice. The case for Shapley rests on saturation and redundancy in the discriminator. Our argument is that saliency fades when the discriminator is confident, which is when steering alignment matters most. The evidence offered so far is indirect: DASP's paper finds gradient-based attributions noisier and less faithful than its Shapley approximation. The saliency-swap ablation answers the question directly. If it matches the Shapley gate, we will report that the gating policy, not the game theory, is the contribution.

Why not just penalize the spectrum with BSP or MRE? These are cheap, and on VisDA they are hard to beat. They treat all singular directions alike, and they cannot tell the difference between a direction that encodes texture and one that encodes a fine class distinction. We run BSP alone, the gate alone, and the two together. If BSP alone equals the gate, the extra machinery is not worth it.

Why not just project the reversed gradient away from the classifier gradient, PCGrad-style? This costs a single dot product and also protects class-relevant directions. However, it acts on one mini-batch's gradient pair and has no memory of which channels carry domain information. SVFair reports that pairwise projection misses global conflict structure in the multi-task setting, though that evidence comes from cooperative tasks rather than gradient reversal. We include feature-level projection as a baseline under the same budget.

Decisive Checks

The experiments run on Office-31, Office-Home, VisDA-2017 and DomainNet, with MNIST→MNIST-M for fast iteration. The gate must beat BSP+CDAN on Office-Home and VisDA-2017. It must also beat the saliency-swap ablation, with most of the margin attributable to the domain-side attribution.

The entangled-channel experiment, comparing $\beta>0$ against $\beta<0$, should produce a clear winner. Ideally the winner depends predictably on how much class structure the target domain shares with the source, for example Product→Real-World versus Clip Art→Product on Office-Home. That would reconcile BIWAA with BSP rather than simply picking a side.

On the estimator side, SVARM at $T=256$ should reach a rank correlation above 0.9 with the Monte Carlo reference on the discriminator game. If it does not, the noise in the scores, rather than the idea, would explain any failure, and DASP becomes the primary estimator. The spectral diagnostic should show that the gate keeps secondary singular values from collapsing, and the gain should be concentrated on the rarest target classes.