When engineers want to know how a machine-translated paragraph "reads," they can ask people to rate it between pairs of opposite adjectives — precise–vague, clear–chaotic — and then use factor analysis, a statistical technique that compresses many highly correlated ratings into a few underlying dimensions, to recover a small, interpretable perceptual space. This is semantic differential (SD) scaling, a psychometric method with a long track record in audio quality, urban soundscapes, and, recently, machine-generated text. When engineers want to know whether a protein will express solubly in a microbial host, they have nothing comparable: they get a single binary verdict — soluble or not — from an opaque model, with no continuous, interpretable coordinate system describing why a construct fails.
We propose to build that coordinate system. Concretely: treat a battery of bipolar biophysical descriptors — each protein scored along opposite poles such as hydrophobic–hydrophilic, ordered–disordered, compact–extended — as the computational analog of an SD adjective-pair battery, apply the same factor-analytic machinery SD scaling uses, and test whether the resulting low-dimensional bipolar space captures the expressibility information inside modern protein language models. The payoff is targeted at two documented failures of current predictors: their blindness to single mutations that flip solubility, and their refusal to transfer across host organisms.
Core Claim
The claim has two parts, one methodological and one scientific. Methodologically, the SD pipeline — curated bipolar anchors, factor analysis with rotation, cross-dataset factor-stability validation — can be transplanted from human perception to protein production phenotypes, with the per-construct "ratings" computed from sequence rather than elicited from panelists, and with a small expert panel doing the curation and validity checking that SD scaling normally demands. Scientifically, we hypothesize that the expressibility-relevant signal in protein language model embeddings is largely contained in a small (roughly four to eight) set of bipolar factors, and that forcing predictions through this factor bottleneck costs little or no accuracy relative to black-box fine-tuning while restoring sensitivity to point mutations and enabling cross-host transfer.
Formally, for each protein construct $s$ we compute a vector $x(s) \in \mathbb{R}^P$ of $P$ bipolar descriptor scores, each zero-centered so that positive and negative values correspond to the two poles. The factor model is the standard SD decomposition:
$$x(s) = \Lambda f(s) + \epsilon(s),$$
where $\Lambda \in \mathbb{R}^{P \times K}$ holds the loadings of each descriptor on $K$ latent factors, $f(s) \in \mathbb{R}^K$ is the construct's factor profile, and $\epsilon$ is descriptor-specific residue. Expression outcome is then predicted as $y(s) = g(f(s))$ with a small classifier head, and separately we probe whether $f(s)$ can be recovered from language-model embeddings, $f(s) \approx h(e(s))$.
We are deliberate about the limits of the analogy. SD scaling in its home setting captures human perception; here the "perception" is a computed biophysical profile. What we transfer is the part of the method that carries its value: bipolar anchoring (which forces every axis to have a meaningful negative pole, unlike one-sided feature importance scores), factor compression of collinear descriptors, and the validation discipline of checking that the same factors re-emerge in independent stimulus sets.
Research Program
Three components. First, we curate the bipolar descriptor battery. The raw material already exists: Sequence-Based Prediction of Protein Solubility profiles 28 physicochemical properties over sliding windows and shows that just six — disorder/coil, hydrophobicity, hydrophilicity, β-turn and α-helix propensity — carry most of the solubility signal on the Niwa et al. E. coli proteome measurements; SoluProt: prediction of soluble protein expression in Escherichia coli uses 251 sequence-derived features. We convert these into signed bipolar pairs, have a panel of roughly twenty protein-production experts curate the anchors following SD practice, and discard descriptors that fail discriminant validity — exactly as Perceptual Quality Dimensions of Machine-Generated Text with a Focus on Machine Translation dropped fluent–non-fluent for poor discriminant validity among its twenty pairs.
Second, we run exploratory factor analysis with oblique rotation, mirroring the maximum-likelihood PROMAX procedure that resolved four dimensions (Precision, Complexity, Grammaticality, Transparency) from MT text ratings. The precedent for expecting stable low-dimensional structure across independent datasets is Underlying quality dimensions of modern telephone connections, where telephone-quality ratings repeatedly compress into the same three dimensions. Our analogous check: do the same protein factors emerge from the E. coli NESG pipeline data and the PDBSOL corpus independently?
Third, we couple the factor space to deep representations. Deep protein representations enable recombinant protein expression prediction showed UniRep embeddings implicitly capture solubility and hydropathy but never decomposed them; ProtSAE: Disentangling and Interpreting Protein Language Models via Semantically-Guided Sparse Autoencoders disentangled ESM2-15B activations into discrete Gene Ontology concept neurons. We differ from ProtSAE in two ways: our units are continuous bipolar axes rather than binary concept neurons, and our target phenotypes are production outcomes (inclusion body formation, soluble yield) rather than functional annotations. We learn $h$ mapping embeddings to factor scores, measure how much embedding variance the factor space explains, and quantify any residual "dark matter" — embedding signal orthogonal to every interpretable factor yet still predictive of outcome.
Proposed Experiments
Training and evaluation use the curated solubility corpora in the file: the PSI: Biology set (tag-leak cleaned, per the shortcut-learning critique in NetSolP: predicting protein solubility in E. coli using language models), the Price independent test set, the NESG-SoluProt set of 9,644 proteins, and PDBSOL with its more than 60,000 sequences. Metrics are AUC, MCC, and accuracy on the binary solubility labels; factor quality is measured by explained variance, factor congruence (Procrustes correlation) across datasets, and the correlation between factor-score shifts and measured solubility changes. For the mutation-sensitivity test we use point-mutation solubility data of the CamSol kind that NetSolP documented as a failure mode for transformer models — the file cites no reported number on that data, so we establish our own reference. Cross-host transfer is tested against the Novozymes B. subtilis corpus of 4,487 genes used by Deep protein representations enable recombinant protein expression prediction, contingent on access, since it is proprietary; if unavailable, we report this as a limitation and substitute a public B. subtilis set if one can be assembled.
Baselines
The naive reference point is a single-axis predictor: mean hydropathy alone fed to logistic regression, plus the majority-class rate. It tells us how much of the problem one bipolar axis already solves, and it anchors the bottom of every comparison.
The strongest existing method is NetSolP, which fine-tunes ESM and ProtT5 transformers for E. coli solubility and reports AUC 0.73 (MCC 0.29) on the curated PSI: Biology set and AUC 0.76 on the independent Price set (NetSolP: predicting protein solubility in E. coli using language models). This is the number to beat or match. The comparison is run with the same embeddings, the same train/test splits, and the same tuning budget via nested cross-validation, so the only difference is whether prediction passes through the interpretable factor bottleneck or a free-form classification head. Beating it means matching its AUC within confidence intervals on both benchmarks while additionally winning on mutation sensitivity and producing an explicit factor space; if the bottleneck costs more than about two AUC points, the interpretability claim is not worth the accuracy loss and we say so.
The interpretable-features champion is SoluProt, a gradient boosting machine over 251 handcrafted descriptors that reports 58.5% accuracy and AUC 0.62 on the independent NESG set (SoluProt: prediction of soluble protein expression in Escherichia coli), notably outperforming some deeper architectures there. It matters because it is the closest spiritual cousin to our proposal — interpretable features, classical learner — and beating it isolates what the factor structure adds over raw features. We also run PROTSOLM's multimodal setting (accuracy 0.794 on PDBSOL-test, PROTSOLM: Protein Solubility Prediction with Multi-modal Features) as a context comparison, though its benchmark differs.
Two ablations isolate the key ingredient. One removes the factor analysis: the same bipolar descriptor battery goes straight into the same classifier head, unreduced. The other keeps the dimensionality reduction but replaces EFA with plain PCA at matched dimensionality, discarding bipolar anchoring. If the EFA version beats both, the win is attributable to the interpretable factor structure itself, not merely to compression.
Why not just…?
Why not just fine-tune a protein language model end-to-end? This is the real threat, and it is exactly what NetSolP does, with reported AUC of 0.73–0.76. Our design makes the head-to-head the central experiment rather than an argument: same embeddings, same data, bottleneck versus no bottleneck. The documented weakness of the end-to-end route — insensitivity to single point mutations that alter aggregation, as NetSolP itself found on CamSol-style data — is precisely where a continuous bipolar factor space should help, because one mutation moves a factor score smoothly instead of having to flip a class. If fine-tuning wins on both accuracy and mutation sensitivity, the proposal is refuted and we will report it that way.
Why not just use the 251 handcrafted features with a gradient boosting machine? Because that is nearly an ablation of our method with the central ingredient removed, and its measured ceiling — AUC 0.62 on NESG (SoluProt) — sits well below the transformer results. Our bet is that bipolar factor compression plus the embedding-to-factor mapping adds two things raw features lack: a validated, cross-dataset-stable latent structure that transfers across hosts, and a coupling to deep representations that recovers signal the handcrafted battery misses. If the factor head does no better than the raw-feature head at matched capacity, our claim collapses to SoluProt with extra steps, and the experiments are built to expose exactly that.
Why not just fix hard proteins empirically — fusion tags, secretion signals, codon optimization? These are real and effective: the artificial XXA fusion tag rescued soluble expression of the SARS-CoV-2 protease TMPRSS2 without any modeling (Improving Soluble Expression of SARS-CoV-2 Spike Priming Protease TMPRSS2 with an Artificial Fusing Protein), the FAS2FURIOUS pipeline routes difficult proteins through secretion in Drosophila S2 cells (FAS2FURIOUS: Moderate-Throughput Secreted Expression of Difficult Recombinant Proteins in Drosophila S2 Cells), and CodonTranslator: a conditional codon language model for codon optimization across life domains optimizes codon usage across 2,163 species. But all of these operate after you have chosen a construct and a host; none tells you which candidates to attempt, which host to pick, or which factor pole is causing failure. They are complementary, and our factor profiles are what would tell an engineer which rescue to try — a hydrophobic-exposure problem suggests a shielding tag, a disorder problem suggests a secretion route. The prioritization value is already demonstrated in miniature: the UniRep B. subtilis model's candidates predicted at ≥80% expression probability achieved over 80% actual precision (Deep protein representations enable recombinant protein expression prediction).
Decisive Checks
Four results would decide the proposal. First, the bottleneck predictor must match NetSolP's reported AUC of 0.73 on PSI: Biology and 0.76 on Price within confidence intervals. Second, factor-score deltas must correlate with mutation-induced solubility changes more strongly than black-box logit deltas do; no reported result exists on Lacuna for mutation-level solubility benchmarks, so this comparison establishes the first reference numbers. Third, factors extracted from E. coli data must show high congruence when re-estimated on independent corpora, echoing the cross-dataset stability that made SD scaling credible in audio quality work. Fourth, refitting only the small outcome head on B. subtilis data must beat direct transfer of an E. coli-trained classifier — which the file records as failing — establishing that the factor space, not just the head, is the transferable object. Any of these failing is a publishable, informative negative; all four succeeding would give recombinant protein production something it currently lacks: a shared, interpretable bipolar language for why constructs fail.
References
- Sequence-Based Prediction of Protein Solubility
- SoluProt: prediction of soluble protein expression in Escherichia coli
- Perceptual Quality Dimensions of Machine-Generated Text with a Focus on Machine Translation
- Underlying quality dimensions of modern telephone connections
- Deep protein representations enable recombinant protein expression prediction
- ProtSAE: Disentangling and Interpreting Protein Language Models via Semantically-Guided Sparse Autoencoders
- NetSolP: predicting protein solubility in E. coli using language models
- PROTSOLM: Protein Solubility Prediction with Multi-modal Features
- Improving Soluble Expression of SARS-CoV-2 Spike Priming Protease TMPRSS2 with an Artificial Fusing Protein
- FAS2FURIOUS: Moderate-Throughput Secreted Expression of Difficult Recombinant Proteins in Drosophila S2 Cells
- CodonTranslator: a conditional codon language model for codon optimization across life domains