Context-Gated Pairwise Potentials for Joint PTM Decoding

Proteins are decorated after synthesis with chemical tags such as phosphate, acetyl, methyl, or ubiquitin groups. These tags interact: one tag can block, prime, or compete with another. This interplay is called post-translational modification (PTM) crosstalk. Current predictors still score each residue and each tag type more or less independently, and then read correlations off the outputs. We propose to predict all candidate tags on a protein jointly with a conditional random field (CRF), a model that scores a whole labeling at once instead of one label at a time. In our CRF, the strength of each pairwise coupling between two candidate tags is computed from continuous context: how far apart the two residues are in sequence and in 3D, and whether they sit in a floppy, intrinsically disordered region. The model can then say "if K12 is acetylated, the chance that S15 is phosphorylated drops." It can also condition on tags that mass spectrometry has already observed and complete the rest of the pattern.

The design choice is borrowed from speech research, where prosody (pitch, rhythm, and pauses laid over the words) is a continuous side-signal that changes how word-level decisions should combine. The lesson from that field is that where the side-signal enters the model matters more than whether it is present. Hear2Act found that handing multimodal LLMs raw audio alongside transcripts moved the optimal-decision rate only from 14.6% to 15.3%. Routing prosody through an explicit intermediate state raised it to 39.6%, close to the 40.7% ceiling. In the same vein, SPECTRA shows through ablation that removing its explicit token–audio alignment objective weakens cross-modal attention and lowers accuracy. Our analogous hypothesis concerns proteins. Structural context fed into per-site features (the "raw audio" route) is underused for crosstalk. Structural context that directly sets the pairwise couplings (the "explicit state" route) is what lets the model represent crosstalk.

Core Claim

A joint model over all (residue, PTM type) variables of a protein, whose pairwise potentials are functions of continuous structural context, will beat per-site multitask predictors at one specific task: completing a protein's modification pattern from a partial observation. The advantage should come from the context-gated couplings themselves, not from extra capacity. If that holds, the learned couplings become a readable, testable map of candidate crosstalk pairs.

Concretely, let each node $i$ be a binary variable $y_i \in \{0,1\}$ indicating whether PTM type $\tau_i$ occurs at residue $r_i$. The score of a full labeling $\mathbf{y}$ of protein sequence $\mathbf{x}$ is

$$s(\mathbf{x},\mathbf{y}) = \sum_i \theta_i(\mathbf{x})\, y_i + \sum_{(i,j)\in E} W_{\tau_i \tau_j}(\mathbf{c}_{ij})\, y_i y_j .$$

Here $\theta_i$ is a per-site score from a frozen protein language model encoder. $E$ links nodes on the same residue (competition) and nodes on residues within a sequence window or 3D radius (long-range crosstalk). $\mathbf{c}_{ij}$ is a vector of continuous context: sequence separation, 3D distance, local disorder of both residues, and local net charge. $W_{\tau\tau'}(\cdot)$ is a small network, one output per type pair, that maps context to a coupling that can be positive (cooperation) or negative (antagonism).

Relation to Prior Work

Several papers motivate the gap. Biology repeatedly shows that inter-residue crosstalk depends on context. In IκBα, ordered phosphorylation and sumoylation decide between degradation and stabilization (PTMs of IκBα). Spindlin1 reads the combined pattern of an activating and a repressive histone mark (Spindlin1 readout). Disordered cytokine-receptor tails pack overlapping short motifs that support graded rather than all-or-nothing signaling (structural disorder in cytokine receptors). The genomics side points the same way: ActiveDriver's analysis finds cancer mutations enriched at lysines used for both acetylation and ubiquitination.

Four computational methods are the competitors, and they model crosstalk only implicitly or statically:

  • MethylSight 2.0 (multitask lysine methylation) shares one encoder across competing lysine modifications. Any correlation it captures lives only in that shared encoder.
  • COMPASS-PTM (mechanism-aware PTM code) initializes a crosstalk prompt matrix from co-occurrence statistics. That matrix is fixed per type pair and does not depend on where the residues are.
  • MeToken (micro-environment tokens) encodes 3D neighborhoods, but uses them as per-site features.
  • PTM-Mamba (PTM-aware language model) accepts PTM tokens as input. It can therefore condition on known tags and is our most serious competitor.

None of these defines an explicit joint distribution whose pairwise terms depend on context.

The method side reuses CRF machinery. Jagannatha & Yu made CRF transition potentials depend on context through convolutional networks and approximated long-range "skip" edges in clinical text. We carry over both ideas, swapping word context for structural context. Our graph has loops, so exact Viterbi decoding is unavailable. We use unrolled mean-field inference, a fixed number of parallel message-passing steps run on the GPU. This follows the spirit of Bregman CRFs, which replaced serial dynamic programming with parallel iterative projections.

Research Program

Data. We build per-protein label sets on the MeToken sequence-structure dataset of 1.26 million sites across 72 PTM types. Splits are by homology-clustered protein, so no protein family is shared between train and test. Context features come from the structures in that dataset and from per-residue disorder estimates.

There is a real labeling caveat. Database annotations pool many cells and conditions, so two tags listed on one protein may never co-occur on the same molecule. For a cleaner test set we use proteoform-level entries from iPTMnet, which link specific modification combinations to individual protein forms.

Training. We maximize pseudo-likelihood plus a mean-field-unrolled likelihood. Unmodified sites vastly outnumber modified ones, so we handle that imbalance by reweighting the positive class.

Partial-observation training. During training we randomly clamp a fraction of true labels. This teaches the model to complete a modification pattern from partial evidence, which is the practical setting of a targeted mass spectrometry follow-up.

Coupling analysis. We will check whether the learned couplings behave as biology predicts:

  • They should decay with 3D distance.
  • They should be stronger inside disordered regions.
  • They should recover the known IκBα and histone H3 K4/K9 relationships without those cases being specifically supervised.

Baselines

Naive reference points. The first is a per-type frequency prior. The second is an independent per-site logistic head on frozen ESM-2 embeddings. All other models must clear both by a wide margin to count.

MethylSight-style multitask model. We reimplement the MethylSight 2.0 recipe on the same ESM-2 backbone, splits, and hyperparameter search budget. It is the strongest shared-representation account of crosstalk. The file reports 0.672 AUPRC for methylation (MethylSight 2.0), but on its own data. We will rerun it on our splits rather than compare across datasets.

MeToken. It is the strongest structure-as-features model, with a reported 50.40% F1 across 72 PTM types on its dataset (MeToken). Because we train on that same dataset, a direct comparison is possible. Beating MeToken means exceeding its macro-F1 on marginal per-site prediction under the same protocol, while also winning on conditional completion.

PTM-Mamba-style conditioner. It is the strongest conditioning competitor: a model fine-tuned to take observed PTM tokens as input and predict the hidden ones, matched to our parameter count and training budget. We found no reported result on Lacuna for PTM-Mamba on a partial-observation completion task, so this comparison is new.

COMPASS-style co-occurrence prior. A fixed type-pair matrix taken from co-occurrence statistics, used as the CRF's pairwise term.

Ablations isolating the key ingredient. Three variants share the backbone and inference procedure:

  • a CRF whose couplings are static per type pair, with no context;
  • a CRF with context concatenated into the per-site scores $\theta_i$ only;
  • the full model, with context gating the couplings $W$.

The central claim requires the full model to beat both other variants on conditional AUPRC at equal parameter count.

Why not just…?

Why not just use a multitask predictor with a shared encoder? A shared encoder can learn that two types correlate on average. It cannot represent "these two particular sites antagonize each other because they are 6 Å apart in a disordered loop." Nor does it give a principled way to update predictions when one site is observed. Marginal AUPRC may well tie. The decisive measure is conditional completion: if clamping observed labels does not improve our model's predictions on the hidden ones more than it improves a retrained multitask model given those labels as inputs, the claim fails.

Why not just feed structure features into the per-site predictor, as MeToken does? This is exactly the "raw audio alongside transcript" route that Hear2Act found nearly inert for changing decisions. Proteins may behave differently, which is why the emissions-only ablation is in the design. If it matches the full model, the prosody-derived hypothesis is wrong for this domain, and we will report that.

Why not just let a PTM-aware language model condition on observed tags? This is a genuine threat: PTM-Mamba already takes modification tokens as input. Given enough data, a masked model could match our completion accuracy. Our model has two concrete advantages. First, its couplings are explicit, so it produces readable crosstalk hypotheses. Second, it shares pairwise parameters across contexts, which should help sparse PTM types such as lactylation and crotonylation. We will test that by reporting completion accuracy stratified by type frequency. If the conditioner ties even on rare types, we will present the CRF as an interpretable equal rather than a stronger predictor.

Why not just reweight independent predictions with a co-occurrence prior? Co-occurrence statistics ignore geometry and are polluted by the pooling caveat above. The static-coupling ablation measures exactly how much the context gate adds over this prior.

Decisive Checks

Primary metric. Conditional AUPRC on hidden sites when 25% or 50% of a test protein's labels are clamped, evaluated on homology-held-out proteins and separately on iPTMnet proteoform entries. Secondary metrics are marginal macro-F1 and AUPRC per PTM type.

Kill criteria. The idea is refuted if either of the following holds:

  • The emissions-only or static-coupling CRF comes within noise (three seeds) of the full model.
  • The matched PTM-Mamba-style conditioner equals the full model on rare types.

Supporting evidence. The idea is supported if the learned couplings both:

  • weaken as 3D distance grows and strengthen inside disordered regions, independent of sequence separation;
  • recover held-out literature crosstalk pairs such as IκBα's phosphorylation–sumoylation priority at rates above the co-occurrence prior.

A final exploratory check asks whether the residues our model ranks as top crosstalk sites show the cancer-mutation enrichment that ActiveDriver reports for dual-use lysines.