Paying for Pareto Search Only Where RNA Fails to Predict Protein

When a model predicts single-cell protein levels from RNA, it has to get two things right. The first is each protein's own distribution: CD4 has two peaks, and CD45RA has a long tail. The second is how proteins co-vary across cells. The strongest current translator, scChord, fits a probabilistic autoencoder to RNA and then learns a flow, a learned transport that moves each cell's RNA embedding to a protein state. It recovers the individual distributions well. Its authors concede, however, that as the protein panel grows to hundreds of markers the predicted correlation structure degrades. In computer vision a similar conflict, between pixel fidelity and perceptual realism, was handled by EA-Adam. That method evolves a population of networks along a trade-off curve and interleaves evolutionary moves with gradient steps. The cost was roughly 3x the training time (144 versus 48 hours), which its authors name as their main limitation.

This proposal asks whether that price is necessary for RNA-to-protein translation, and argues that most of it is not. The conflict between matching each protein's distribution and matching the joint correlation structure should be concentrated in discordant proteins, meaning those whose abundance is poorly explained by their own transcript. Mass-spectrometry surveys report mRNA–protein agreement only 40–60% of the time (A high-stringency blueprint of the human proteome), so such proteins are common. For concordant proteins the two objectives should agree, and plain gradient descent suffices. If this holds, the population can be replaced by a single network that takes a trade-off preference as input, with that input applied only to the discordant proteins' output heads. This network would trace the full trade-off curve in roughly one training run.

Core Claim

We make two claims, each testable on its own. The first is diagnostic. On a large CITE-seq panel (CITE-seq measures RNA and surface-protein counts in the same cells), the gradients of a per-protein distribution-matching loss and a correlation-matching loss point in conflicting directions mostly for proteins with low self-predictability. Self-predictability is the held-out $R^2$ of predicting a protein from its own gene's mRNA alone. The second claim is methodological. A preference-conditioned translator, whose conditioning is restricted to those conflicted proteins, recovers a trade-off curve at least as good as a population-based hybrid optimizer and a sweep of separately trained models, at about one model's training cost. It must also not sacrifice cell-level accuracy.

We measure correlation structure with the Correlation Matrix Distance used in the scChord evaluation:

$$\mathrm{CMD}(R, \hat R) = 1 - \frac{\mathrm{tr}(R \hat R)}{\|R\|_F \, \|\hat R\|_F},$$

where $R$ and $\hat R$ are the observed and predicted protein–protein correlation matrices and $\|\cdot\|_F$ is the Frobenius norm. We compute it both globally and within each annotated cell type. The within-type version is essential. Global correlations are dominated by cell-type composition, which any decent translator captures for free. The biologically meaningful and hard part is co-variation among cells of the same type.

What is new is the connection between the two directions. The expensive part of hybrid optimization, maintaining many candidate solutions along a trade-off, is justified only where objectives genuinely conflict. In this problem, where they conflict is predicted by a measurable biological property, discordance.

Research Program

The backbone is a generative RNA-to-protein translator. Our primary backbone is a reimplementation of scChord's two-stage design: an autoencoder with a zero-inflated negative binomial decoder (a count model that absorbs dropout and overdispersion), followed by flow matching. As a secondary backbone we use a totalVI-style joint model, whose protein decoder separates background from foreground counts. Using two backbones checks that conclusions are not specific to one architecture.

Both objectives are made differentiable on minibatch samples. The marginal objective is the average over proteins of the one-dimensional Wasserstein-1 distance, computed by sorting predicted and observed values within cell-type strata. The joint objective is within-type CMD computed on the batch. For the flow backbone we compute both on samples from a short fixed-step integration during a fine-tuning stage, and we report the added cost.

The first analysis is the conflict map. For each protein $j$ we compute self-predictability $s_j$, and during training we record the cosine between the gradients of the two losses with respect to that protein's output head. The prediction is a positive rank correlation between $s_j$ and gradient agreement, with the most negative cosines falling on low-$s_j$ proteins. If this relationship is absent, the localized design is unjustified. We would report that result and fall back to global conditioning.

The method, when the conflict map supports it, is a single translator whose protein heads for the conflicted set $C$ take a preference $\lambda \in [0,1]$ as an extra input. Here $C$ is defined as the proteins with $s_j$ below a threshold chosen on validation data. Training samples $\lambda$ per batch and minimizes

$$\mathcal{L}(\lambda) = (1-\lambda)\, W_1^{C} + \lambda\, \mathrm{CMD}_{\text{type}} + W_1^{\bar C} + \mathcal{L}_{\text{base}},$$

where $W_1^{C}$ and $W_1^{\bar C}$ are the marginal losses on conflicted and non-conflicted proteins, and $\mathcal{L}_{\text{base}}$ is the backbone's own likelihood objective. The shared trunk and the concordant proteins' heads are trained by ordinary gradient descent with no preference input. At inference, varying $\lambda$ traces the curve with no retraining. This is the step that removes the population. Instead of $N$ networks trained concurrently, as in EA-Adam or in evolutionary reinforcement learning (Majumdar et al.), one network amortizes the curve.

We will not attempt evolutionary gene-subset selection or evolved interaction graphs. The nested outer-search/inner-training pattern of SHAREs and of swarm feature selection (RSO, tested only up to 309 features) scales as candidates times inner iterations. At 20,000 genes and 100,000+ cells that is the wrong place to spend compute, and sparse input gating would be a cheaper and likely equal competitor. The static-network reconciliation of Huang et al. remains a complementary bulk-level tool. It is not a competitor on single-cell generation.

Baselines

The naive reference point is the backbone trained with its standard objective and no correlation term. This is scChord or totalVI as published, retrained on our splits. It is where the curve's "marginal" end should lie. The file reports that scChord achieved the lowest RMSE and highest Pearson correlation against totalVI, sciPENN and scTranslator. It also reports that scChord trains on 100,000 cells in about 27 minutes using 229 MB of GPU memory. We found no reported Wasserstein or CMD values on Lacuna for scChord on the 228-protein PBMC panel, so we will establish these numbers ourselves under identical splits.

The strongest cheap competitor is a scalarized sweep: $K$ separately trained backbones, each fine-tuned with a fixed weight $\lambda_k$ on the same combined loss, with $K$ chosen to match or exceed our compute. It is the strongest baseline because each model specializes to one point on the curve.

The strongest hybrid competitor is EA-Adam from Sun et al.. It evolves a population of translators by crossover and mutation interleaved with gradient steps, using the two objectives, and then fuses weights across the population. Its reported numbers are from image super-resolution (on Urban100, over 0.4 dB PSNR gain and over 10% LPIPS reduction versus Adam, at about 3x training time). There is no reported result on Lacuna for EA-Adam on any CITE-seq benchmark.

A post-hoc copula baseline takes each protein's predicted marginal from the naive model and re-orders samples within cell type to match the training rank-correlation structure.

For the ablation that isolates the key ingredient, the conditioned model is trained with preference input on all protein heads rather than only the conflicted set. It is also compared with a variant whose conditioned set is chosen at random but matched in size.

All methods share data splits, preprocessing (the same highly variable genes), backbone architecture, and a hyperparameter budget counted in GPU-hours, including EA-Adam's population cost. Beating the baselines means reaching equal or higher hypervolume, the area of objective space dominated by a method's curve with a fixed reference point, at no more than 1.5x the naive model's GPU-hours. Cell-level Pearson correlation must stay within one standard error of the naive model across five seeds. The sweep and EA-Adam are allowed several times more compute.

Why not just…?

Why not just sweep the loss weight and train several models? This is the real threat. At 27 minutes per scChord run, a ten-point sweep takes about 4.5 hours, so raw overhead is not prohibitive on this backbone. The proposal's case therefore rests on three things. First, hypervolume per GPU-hour. Second, a continuous curve that a user can move along at inference for each cell type. Third, the conflict map, which is informative regardless of optimizer. If the sweep matches our hypervolume at comparable total cost, we will report that hybrid population methods and amortization are unnecessary here. That would still be a useful negative result, given how often population search is proposed for such trade-offs.

Why not just add a correlation penalty to scChord at one well-chosen weight? A single weight is one point of the sweep and inherits its analysis. We include it as the sweep's best-validation member. If the curve turns out to be nearly flat, meaning the correlation term costs almost no marginal fidelity, then the "trade-off" is an artifact of never including the term. One weight would suffice, and the conditioned model adds nothing. The hypervolume comparison exposes this directly.

Why not fix correlations after the fact with a copula reordering? Reordering can drive within-type CMD toward the training structure without touching marginals. Its weakness is that it ignores each cell's RNA when deciding which cell gets which protein value, so it should damage cell-level accuracy. We make cell-level Pearson correlation and RMSE a mandatory third axis for exactly this reason. If the copula improves CMD without hurting cell-level accuracy, the problem is not an optimization problem at all, and we would say so.

Why not just use the population optimizer that already worked in vision? EA-Adam's advantage was attributed to escaping stagnation of a single gradient trajectory. It paid 3x in time for it. Our conflict-map hypothesis predicts that in this problem the stagnation is confined to a subset of output heads, so evolving whole networks mostly spends compute where objectives agree. Running EA-Adam at matched compute tests this. If it beats the localized conditioned model, the localization hypothesis is wrong.

Proposed Experiments

The primary benchmark is the PBMC atlas (GSE164378, over 161,000 cells, 228 proteins), where scChord reports correlation-structure degradation. Secondary benchmarks are the murine spleen and lymph node data with 111- and 208-protein panels used by totalVI. The CBMC data (about 8,600 cells, 13 proteins) serve as a negative control, where few proteins should be conflicted and all methods should coincide. Splits follow the published 70/30 protocol used by PIKE-R2P on CBMC. For PBMC we use donor-held-out splits to avoid leakage between cells of the same donor. Metrics are per-protein Wasserstein-1, within-type and global CMD, cell-level RMSE and Pearson correlation, hypervolume, wall-clock time and peak memory.

The first decisive check is the conflict map. The claim survives only if gradient agreement rank-correlates positively with self-predictability on PBMC, replicates on both murine panels, and weakens on CBMC.

The second decisive check is the matched-compute comparison. The conditioned model must reach at least the sweep's hypervolume at under half the sweep's GPU-hours and must not lose to EA-Adam. The localized ablation must beat both all-heads conditioning and random-subset conditioning. If localization only ties all-heads conditioning, the discordance link is decorative and we drop it.

The third check is biological. We ask whether moving $\lambda$ toward the correlation end restores known within-type co-variation, such as CD45RA and CD38 among CD8 T-cell subsets, which scTEL used to separate naive from effector-memory cells. We also ask whether it does so without flattening CD4's two peaks.