When two amino acids touch inside a folded protein, evolution tends to mutate them together. That is the basis of direct coupling analysis (DCA), a family of statistical methods that reads residue contacts off an alignment of related protein sequences. The hard part is not detecting co-variation. It is "explaining away" the indirect kind. If A touches B and B touches C, then A and C co-vary even though they are far apart, and a method has to subtract out these chains of influence. Classical DCA does this with a global computation: inverting a covariance matrix, or fitting a full pairwise model. Modern neural contact predictors mostly run a single forward pass, either over pre-inverted couplings or directly over the alignment. Work on recurrence in vision models argues that fixed-depth, single-pass networks fail on exactly the inputs that need iterative reinterpretation.
This note proposes to test whether the feedforward constraint binds in contact prediction, and to do it with the same controlled methodology vision researchers used. We build a pair-representation module whose weights are shared across iterations. Each iteration combines pair features along the shared residue index, which is the operation matrix inversion needs. The module halts when its contact map stops changing. The specific claim has three parts. First, the iterations of this module behave like a regularized iterative solver for the inverse problem. Second, it should beat feedforward stacks matched in parameters and in compute, and the advantage should concentrate on protein families with long indirect chains and on shallow alignments. Third, its learned stopping time should track how ill-conditioned the alignment's covariance is. If none of this holds, we will have shown that one-pass networks are not limited here, which is also worth knowing.
Core Claim and Why It Is Not Already Done
The mechanism comes from Gaussian DCA. Fast and Accurate Multivariate Gaussian Modeling of Protein Families treats one-hot sequences as Gaussian variables and reads direct couplings off the inverse covariance, roughly $J = -C^{-1}$. Here $C$ is the regularized empirical covariance of alignment columns. Write $C = D(I - A)$, where $D$ holds the single-site variances and $A$ the normalized off-diagonal correlations. Then
$$C^{-1} = \Big(\sum_{k \ge 0} A^k\Big) D^{-1},$$
which converges when the spectral radius $\rho$ of $A$ is below one. The $k$-th term subtracts correlation transmitted along chains of $k$ contacts. The number of terms needed for a fixed error grows roughly like $1/(1-\rho)$, which follows from geometric convergence at rate $\rho$.
This picture makes two points. Explaining away is a multi-hop computation that combines pair $(i,k)$ with pair $(k,j)$. That is a contraction over a shared index, which a local 2D convolution cannot express. And with shallow alignments $C$ is badly conditioned, because related sequences share recent ancestry (Monasson et al.). The theory of iterative solvers says that stopping such an iteration early acts as regularization. So an input-dependent iteration count is not just a compute saving. It is a depth-dependent regularizer.
The closest existing work falls short in specific ways. plmConv (Protein contact prediction from amino acid co-evolution using convolutional networks for graph-valued images) runs a three-layer feedforward convolutional network on couplings that plmDCA has already inverted. It outsources explaining away to the external fit. Co-evolution Transformer (Co-evolution Transformer for Protein Contact Prediction) and the MSA Transformer learn directly from the alignment, but they are fixed-depth stacks with untied layers. The tied row attention in the MSA Transformer shares one attention map across sequences, not across depth.
AlphaFold (Highly accurate protein structure prediction with AlphaFold) is the real precedent. Its recycling feeds the whole trunk's output back as input for a fixed small number of passes, and pair features bias attention over the alignment. Top-down feedback from the contact domain to the sequence domain therefore already exists. For that reason we do not pursue the obvious idea of "add feedback from the contact map to the alignment." Our proposal differs from AlphaFold in three ways. The recurrence is local and weight-tied on the pair representation rather than a coarse loop over the whole network. Its stopping is adaptive. And, most importantly, it is tested against the matched controls that recycling has never been isolated against.
That control logic comes from vision. Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition compared bottom-up, lateral, and top-down recurrent variants against feedforward nets with wider maps or larger kernels. Recurrent neural networks can explain flexible trading of speed and accuracy in biological vision added an unrolled control with independent weights per step and entropy-based halting. Evidence that recurrent circuits are critical to the ventral stream's execution of core object recognition behavior supplies the idea of a "challenge" input class, defined in advance, where one-pass models fail. These papers supply our method and our evaluation logic, not our domain.
Research Program
The recurrent module. The module takes per-pair features computed from a subsampled alignment: raw covariance blocks, without any external DCA inversion. It applies one weight-tied block $T$ for $t$ steps, $z^{(t+1)} = z^{(t)} + T(z^{(t)}, z^{(0)})$. Feeding $z^{(0)}$ back in at every step keeps the evidence available and stabilizes long unrolls. The block contains three parts:
- a contraction over the shared residue index, $\sum_k f(z_{ik})\, g(z_{kj})$, similar to AlphaFold's triangle updates;
- a small convolution for local contact patterns such as helix and strand textures, which plmConv showed matter;
- normalization.
A per-protein halting head stops iteration when the change in the predicted contact distribution falls below a threshold. We train with randomized unroll lengths so the module does not overfit to one depth. The only output is a contact map, because the question concerns contact inference, not full structure prediction.
Two diagnostics. The program has a synthetic arm and a real-protein arm, each aimed at a different question.
The synthetic arm uses adabmDCA 2.0 to fit Potts models to real families, and also to build Potts models with designed contact graphs whose chain lengths and spectral radius $\rho$ we control. We then sample alignments of chosen depth from these models. This gives exact ground-truth direct couplings. It lets us measure accuracy as a function of indirect-chain length and $\rho$, separately from alignment depth.
The real-protein arm defines challenge families in advance, before any model is trained, as families where the raw covariance ranking and the plmDCA ranking of top contacts disagree most. These are the families where explaining away does the most work. In Kar et al.'s terms, they are the challenge images.
Analysis. We test whether the module's intermediate iterates track the partial Neumann sums of the Gaussian DCA inverse. Concretely, we check whether the correlation between $z^{(t)}$ and the $t$-term truncated inverse rises with $t$. We also test whether halting time correlates with the condition number of the alignment covariance and with alignment depth. The theory does not fix the sign of the depth relationship. Early stopping as regularization predicts fewer iterations for noisy shallow alignments. Long chains predict more. The learned behavior tells us which effect dominates.
Baselines
The naive reference point is the raw-covariance or Gaussian DCA ranking with no learning. On deep alignments, Gaussian DCA reports a top-10 contact true-positive rate of about 95% across 50 Pfam families (Fast and Accurate Multivariate Gaussian Modeling of Protein Families). We rerun it on every depth stratum with its own pseudo-count tuned on validation data. The recurrent module must beat it clearly on shallow strata, and must not lose to it on deep ones.
The strongest feedforward-on-couplings competitor is plmConv. It is the right competitor because it hands explaining away to an exact external fit and then learns local contact patterns. That is precisely the division of labor our claim says a learned iteration can improve on. plmConv reports a top-10 long-range positive predictive value of 0.686 on 89 CASP11 targets, against 0.536 for plmDCA and 0.594 for MetaPSICOV (Protein contact prediction from amino acid co-evolution using convolutional networks for graph-valued images). We retrain a modernized version with the same training set, the same pair-feature dimensionality, and the same hyperparameter-search budget as our model. We then evaluate both models on identical depth-subsampled alignments.
The strongest end-to-end competitors are Co-evolution Transformer and MSA Transformer. CoT reports 51.6% top-L long-range precision on CASP14 free-modeling targets (Co-evolution Transformer for Protein Contact Prediction). The MSA Transformer reports 57.4% unsupervised top-L long-range precision on the trRosetta test set, against 41.1% for the single-sequence model ESM-1b (MSA Transformer). We do not expect to match large pretrained transformers in absolute terms. They serve as reference points, and we retrain a parameter-matched CoT-style trunk from scratch on our data for the fair comparison.
The file reports no depth-stratified precision for any of these methods. It describes degradation qualitatively (Protein Modeling with Shallow Multiple Sequence Alignments), so for shallow alignments no reported result was found on Lacuna. We will produce these numbers ourselves.
Ablations isolating the key ingredient. These follow the vision literature. B-D is an untied feedforward stack of the same block, matched in parameters to our model. B-U is an untied unroll matched in compute, with the same number of block applications and $t$ times the parameters. A wider single block (B-F-like) is matched in parameters. A dilated residual convolution stack with global receptive field, in the style of trRosettaX (Improved Protein Structure Prediction Using a New Multi-Scale Network and Homologous Templates), has no index contraction. Finally, we run our own model with a fixed iteration count and with the contraction removed.
Beating these means three things. Our model must show higher long-range top-L precision on the shallow strata and on challenge families than the best compute-matched control, with gaps beyond seed variance. The gap must shrink on deep, non-challenge families. And removing the contraction must destroy the gain.
Why not just…?
Why not just feed plmDCA or Gaussian DCA couplings into a deep feedforward network, as plmConv does? This is the most serious threat. On deep alignments the external inversion already removes indirect chains, and recurrence may add nothing. We will run every model on both raw-covariance inputs and pre-inverted inputs. If the recurrent advantage disappears on pre-inverted inputs at every depth, the honest conclusion is that recurrence only re-implements inversion. Our claim then shrinks to a cost argument. The claim survives only if the learned iteration beats the "invert, then classify" pipeline on shallow alignments. That is where the inversion's fixed pseudo-count regularization is the bottleneck (Monasson et al.), and where combining denoising with explaining away should matter.
Why not just make the feedforward network deeper or wider? In vision, parameter-matched wide and large-kernel controls lost badly under heavy occlusion: 29.6% error against 9.3% for the recurrent network (Recurrent Convolutional Neural Networks). On ImageNet the margin was smaller, 62.68% against 64.37% top-1 accuracy (Recurrent neural networks can explain flexible trading of speed and accuracy). Neither result transfers automatically to this domain. There is also a theoretical caution. A feedforward stack with contraction layers could in principle compute the inverse in logarithmic depth by repeated squaring of $A$. So depth alone need not be the limit. The B-D and B-U controls on the synthetic chain-length sweep settle whether learned feedforward nets actually find such solutions under noise.
Why not just use AlphaFold-style recycling with more passes? Recycling is coarse recurrence over the whole trunk with a fixed pass count. It has never been compared against compute-matched untied controls. We include "recycle the whole feedforward contact network with shared weights, $t$ fixed" as a baseline. If it equals our fine-grained adaptive module, the conclusion is that iteration matters but its locality does not. That result would still answer the question.
Why not just use single-sequence language models and skip alignments? The MSA Transformer's 57.4% against ESM-1b's 41.1% shows that alignment-based co-variation still carries signal that single sequences lack (MSA Transformer). Our question also concerns how that signal is decoded, which single-sequence models do not address.
Proposed Experiments and Decisive Checks
We train on the trRosetta set of about 15,000 chains, with family-level splits. We evaluate on CASP14 free-modeling targets, on CAMEO targets, and on Pfam families. Each test alignment is subsampled to effective depths of roughly 10, 30, 100, and 1,000 sequences, with the subsampling done identically for all models. The metric is long-range top-L and top-L/5 precision for sequence separation of at least 24 residues, reported per depth stratum and per challenge/non-challenge split, with three seeds per model.
Three checks are decisive. On synthetic Potts data, the recurrent model's advantage over B-U should grow with indirect-chain length and with $\rho$ at fixed alignment depth. If it is flat, the mechanism we claim is not what is happening. On real data, the advantage should concentrate on the challenge families, the counterpart of Kar et al.'s 30 ms delay on challenge images. And learned halting time should correlate with covariance condition number at least as well as with raw alignment depth. If depth alone predicts halting equally well, a simple depth-based rule could replace the halting head, and we will report that.
We will also test whether the intermediate iterates align progressively with truncated Neumann inverses, which tells us whether the network learned the solver. Gains that appear only on pre-inverted inputs, or only without matched compute, will be treated as negative results for the claim.
References
- Fast and Accurate Multivariate Gaussian Modeling of Protein Families: Predicting Residue Contacts and Protein-Interaction Partners
- From Principal Component to Direct Coupling Analysis of Coevolution in Proteins: Low-Eigenvalue Modes are Needed for Structure Prediction
- Protein contact prediction from amino acid co-evolution using convolutional networks for graph-valued images
- Co-evolution Transformer for Protein Contact Prediction
- MSA Transformer
- Highly accurate protein structure prediction with AlphaFold
- Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition
- Recurrent neural networks can explain flexible trading of speed and accuracy in biological vision
- Evidence that recurrent circuits are critical to the ventral stream’s execution of core object recognition behavior
- adabmDCA 2.0 - a flexible but easy-to-use package for Direct Coupling Analysis
- Improved Protein Structure Prediction Using a New Multi‐Scale Network and Homologous Templates