Is 3D Structure Learning Interfaces or Hubs? Popularity Bias in Protein Interaction Prediction

Protein–protein interaction (PPI) predictors take two proteins and output the probability that they physically bind. The field increasingly adds 3D structure, predicted or experimental, on top of sequence, on the argument that binding is decided by surface shape. There is a cheaper explanation for high benchmark scores that nobody in the structural PPI literature seems to have ruled out. Interaction datasets are dominated by a few "hub" proteins that appear in very many positive pairs, often because they are abundant in the cell rather than because they bind unusually well. A model can score well by learning "this protein is a hub" and ignoring the partner entirely. Recommender systems know this failure as popularity bias: a system that recommends bestsellers to everyone looks accurate on historical clicks. That community has worked out both how the bias gets into learned vectors and how to remove it.

This note proposes to carry that toolkit into PPI prediction and ask a pointed question: once hub-driven shortcuts are measured and removed, how much of the advantage of structure-aware models over sequence-only models is left? The work has three parts. The first is a degree-controlled evaluation protocol. The second is a visualization that shows vector magnitude, which is how popularity bias hides in embeddings. The third is a head-to-head comparison of debiasing methods borrowed from recommendation. The payoff is a cleaner estimate of what 3D structure actually contributes, and a debiased training recipe if one turns out to be needed.

Core Claim

We hypothesize that part of the reported accuracy of both sequence and structure PPI models comes from per-protein popularity rather than pairwise compatibility. We further hypothesize that the fraction differs systematically between model families. Models that pass messages over the observed interaction network should be most affected. How Graph Convolutions Amplify Popularity Bias for Recommendation? proves that a node's gradient influence grows roughly as its degree to the 3/2 power, so representations get pulled toward hubs. Pair classifiers built on per-protein embeddings can learn the same shortcut through a different route: a per-protein "promiscuity" offset. This mirrors the finding of Adap-τ that the vector length of frequent items grows with their interaction count, so that inner-product scores come to reflect length rather than direction.

The testable prediction is this. If structure adds genuine interface information, its margin over sequence should persist or widen under degree-matched evaluation. Removing the hub shortcut should also make structural models better at locating the actual binding residues. If instead the margin collapses, the field's structural gains are partly a better hub detector.

Where this sits relative to prior work

Two papers motivate the claim. Quantitative assessment of the structural bias in protein–protein interaction assays showed with classical features and support vector machines that affinity-purification (TAP) data track cellular abundance and yeast two-hybrid (Y2H) data favor disordered proteins. It did not test whether learned predictors exploit these confounders. Towards Precision Protein-Ligand Affinity Prediction Benchmark showed that affinity models memorize the wild-type protein's value instead of reading local changes. That is the same "memorize the entity" shortcut in a neighboring task.

The closest PPI work is Learning the Interaction Prior for Protein-Protein Interaction Prediction. It replaces naive pair heads with a module that builds virtual length-3 paths, reflecting the biological observation that complementarity rather than similarity predicts binding. It also evaluates on breadth-first and depth-first network splits that hold out unseen proteins. Those splits remove memorization of specific test proteins, but they do not control how hub-heavy the negatives are. The paper also states that its head is bounded by the quality of the embeddings beneath it. We keep its splits and add degree control. We also measure whether the embeddings themselves carry a hub signal.

The structural models under test supply the method side. Struct2Graph builds residue graphs from 3D coordinates and uses attention between the two proteins. GearNet pretrains a relational residue-graph encoder on AlphaFold structures. TT3D appends Foldseek structural tokens to language-model embeddings. MULAN adds a torsion-angle adapter to a frozen protein language model. None of these analyzes degree effects.

The debiasing side comes from recommendation, where these tools have only ever been tested on consumer data. MACR models popularity as a direct cause of the score, trains a side branch that sees only the item, and subtracts that branch's effect at inference. CLRec shows that contrastive training with negatives drawn from the data distribution implicitly performs inverse propensity weighting. Adap-τ normalizes vector length and fixes the softmax temperature problem this creates.

One constraint shapes the design. Hubs in biology are not purely artifacts, because a kinase really can bind many substrates. Any debiasing that uniformly penalizes hubs may destroy real signal. This is why we evaluate interface recovery and assay transfer in addition to held-out accuracy.

Research Program

Degree-controlled benchmarks. The first artifact is a set of benchmark variants built on SHS27k, SHS148k and Yeast (from STRING), keeping the breadth- and depth-first splits of L3-PPI. For each test split we construct negatives whose per-protein degree distribution matches that of the positives. A model that only knows "hubs interact" is then at chance by construction. We report precision–recall area and micro-F1 overall, within degree quartiles, and on this degree-matched set. The naive reference point is a degree-only predictor: logistic regression on the two training degrees and their product. Every model's score is read against it.

Magnitude-aware diagnosis. The second artifact makes the hub signal visible. Standard t-SNE or UMAP plots are how Protein language models expose viral mimicry and immune escape inspected protein embeddings, but they operate on neighborhoods and discard vector length. We therefore use a polar layout. The angle comes from a 2D projection of the length-normalized embedding, and the radius is the raw norm, with points colored by training degree. We pair it with two numbers. The first is the rank correlation between each protein's norm and its training degree. The second is the share of variance in pair scores explained by a per-protein additive offset, fitted by regressing the logit on protein identity alone. The layout is a diagnostic lens, not the contribution. The offset-variance statistic is what gets compared across models, and it also applies to models whose pair heads are not inner products.

Debiasing comparison. The third component compares debiasing methods on the same encoders. We use a sequence-only baseline (ESM2 embeddings with the standard head), TT3D-style structural tokens, GearNet embeddings, Struct2Graph, and a network message-passing model of the kind L3-PPI plugs into. On top of each we apply three methods: degree-matched negative sampling during training, a MACR-style counterfactual head, and length normalization with Adap-τ temperature.

The counterfactual head, written for a pair $(a,b)$, is

$$\hat y(a,b) = f\big(h_a,h_b\big) - \lambda\,\big[g(a)+g(b)\big],$$

Here $f$ is the usual pair score from protein embeddings $h_a, h_b$, $g$ is a side branch that sees only a protein's identity embedding and is trained jointly to predict interaction on its own, and $\lambda$ sets how much of that direct effect is removed at test time. This is MACR's natural-direct-effect subtraction, with proteins playing both user and item.

Interface recovery. The fourth component is a check on whether debiasing forces geometry to matter. On the IntAct/PDB curated set of Struct2Graph, whose interface residues are known, we measure how well attention recovers true interface residues before and after debiasing. The reported residue-level sensitivity of that attention is only about 30%. That leaves clear room for an increase to show up if the hub shortcut was crowding out interface learning.

Baselines

The naive reference is the degree-only logistic regression described above. If it approaches a published model's score on the standard splits, that alone is a finding. On degree-matched negatives it should sit at chance, which validates the construction.

The strongest existing PPI method is L3-PPI placed on the best available encoder, since it is the only method in the file designed around a biological interaction prior and evaluated on unseen-protein splits. The file reports that it boosts a network message-passing model by 6.02% on Yeast under breadth- and depth-first splits, and gives a 3.88% gain for another model across nine STRING scenarios. No reported result was found on Lacuna for absolute SHS27k or SHS148k scores, so we will rerun it ourselves.

For structure-based pair prediction, Struct2Graph reports 98.89% balanced accuracy on its curated set. We treat that number as a candidate for degree inflation rather than a target to beat.

Fairness rules apply to every comparison. All methods share splits, AlphaFold-derived structures, encoder checkpoints, and an equal hyperparameter budget, with the same number of search trials and seeds. The debiasing weight $\lambda$ and the temperature are tuned on a degree-matched validation split, never on test. Beating L3-PPI means higher degree-matched precision–recall area on unseen-protein splits, with overall area no more than one standard deviation worse across seeds.

The key ablation swaps structural input for sequence input while holding the debiasing method fixed. The structure-minus-sequence gap on degree-matched data is the quantity the whole proposal exists to estimate. A second ablation sets $\lambda=0$ in a model that still trains the side branch. This separates the effect of subtraction from any regularizing effect of the extra branch.

Why not just…?

Why not just train with degree-matched negatives and skip the counterfactual machinery? This is the real threat, and it may well win. It is simple, needs no extra parameters, and directly removes the shortcut the evaluation targets. We run it as a full competitor. If it matches or beats the counterfactual head on degree-matched accuracy, interface recovery, and assay transfer, we will recommend it and drop the more complex method. In that case the contribution becomes the benchmark and the structure-versus-sequence estimate.

One reason it might fall short is that matching degrees in training data from a single assay still bakes in that assay's abundance bias. The TAP-to-Y2H transfer test below is designed to expose that.

Why not just normalize embeddings to unit length? Normalization removes magnitude-based hub effects in inner-product heads. However, Adap-τ shows that performance then becomes highly sensitive to the softmax temperature. Many PPI heads also use concatenation or neural pair scorers, in which a hub offset can live in any direction, not just in vector length. We include normalization with Adap-τ temperature as a competitor. Our offset-variance statistic will show whether the hub signal survives normalization.

Why not just evaluate on unseen-protein splits, which already exist? Breadth- and depth-first splits stop the model from recognizing specific test proteins. They do not stop it from predicting that high-degree-looking proteins interact, and "hubness" can be read from sequence features such as disorder and length, as Esmaielbeiki et al. found. Comparing the degree-only baseline and each model on the standard and degree-matched versions of the same split measures exactly how much these splits leave uncontrolled.

Why not just use direct low-dimensional distributional embeddings for projection-free visualization? Learning Embeddings into Entropic Wasserstein Spaces avoids projection artifacts, but it requires retraining every encoder in a new geometry with costly iterative optimal-transport solves. It also does not explain how existing models behave. The question here is about the representations the field actually uses, so a cheap magnitude-aware plot plus a scalar statistic is sufficient.

Proposed Experiments

The first decisive check is the gap table: sequence versus structure, before and after each debiasing method, on standard versus degree-matched negatives across SHS27k, SHS148k and Yeast. There are two readings.

The structural story survives if the structure advantage is at least as large on degree-matched data. It is further supported if debiased structural models raise interface-residue sensitivity on the Struct2Graph set above its reported ~30%.

It is weakened if the degree-only baseline explains most of a model's standard-split score, or if the structure advantage shrinks by more than half under degree matching.

The second check addresses the biological worry that hubs are sometimes real. We train on TAP-derived yeast interactions (Gavin et al. and Krogan et al.), which the assay-bias study ties to abundance, and test on Y2H pairs. A method that removes only artifactual popularity should improve this transfer, while one that erases real multi-partner binding should hurt performance on the Y2H hubs. We report both groups separately.

The third check is the cross-family diagnosis. We compute norm-degree correlation and offset variance for every encoder. We expect them to be highest for models that pass messages over the interaction network, as the degree-to-the-3/2 argument predicts, and we expect debiasing gains to track these statistics. If they do not, the recommender-style mechanism does not transfer to proteins, and we will report that as a negative result.