A Pathway-Space Measure of Receptor Subtype Selectivity

Receptors come in families of near-twins. Serotonin receptors, muscarinic receptors, phosphodiesterase isoforms and nicotinic acetylcholine receptor subtypes share a common ancestor, a conserved ligand-binding pocket, and — crucially — overlapping signaling machinery, which is why a drug meant for one subtype routinely hits its siblings and produces dose-limiting side effects. Structural biology has shown that subtype discrimination often hangs on astonishingly small physical margins: in the serotonin receptor 5-HT$_{5A}$, a single bulky glutamic acid at position 6.55 (E305), where the homologs 5-HT$_{1A}$ and 5-HT$_{1B}$ carry small alanine or serine residues, is the key selectivity determinant, and swapping it to alanine shifts ligand sensitivity (Structural insights into the ligand binding and Gi coupling of serotonin receptor 5-HT5A). We therefore measure selectivity today the way structural biology frames it: as a ratio of binding affinities measured in a tube. But side effects are not caused by binding — they are caused by the downstream gene programs a receptor engages, and two perturbations with indistinguishable binding behavior can drive completely different downstream programs (Computationally designed GPCR quaternary structures bias signaling pathway activation).

This proposal argues that receptor subtype selectivity should also be measured where its consequences live: in the cell's transcriptome, read out through RNA sequencing and a new kind of functional enrichment analysis. The core obstacle is that subtype-driven transcriptional differences are exactly the kind of signal classical enrichment pipelines destroy — small, coordinated, low-magnitude shifts across co-regulated gene modules, which univariate differential-expression thresholds truncate before enrichment ever sees them. We propose to build a subtype-resolved enrichment geometry: reference perturbations by maximally subtype-selective tools define, for each subtype, a subspace in gene-expression space; any test ligand's response is then projected onto these subspaces, yielding a transcriptomic selectivity margin — the pathway-space analogue of a binding selectivity ratio — plus a residual that flags cross-reactive signaling no reference program explains. No existing method provides subtype-resolved gene sets or such a decomposition, and the corpus is explicit that this three-way intersection is unoccupied.

Core Claim

The claim has two parts, one methodological and one scientific. Methodologically: if we build gene-set libraries whose "sets" are the expression signatures of defined subtype-selective reference perturbations, rather than the isoform-agnostic pathways in standard databases, then principal-angle enrichment — the geometry introduced by the Characteristic Direction method (The characteristic direction: a geometrical approach to identify differentially expressed genes) — can decompose a ligand-induced transcriptomic response into per-subtype components. Scientifically: this transcriptomic selectivity margin will (i) separate ligands targeting sibling subtypes that share a binding pocket, and (ii) supply the missing labels and features that today's chemical-descriptor machine learning lacks, most visibly its failure on the rare, valuable class of genuinely selective compounds.

The geometry is simple enough to state in one line. For a ligand $\ell$, let $\mathbf{v}_\ell$ be its Characteristic Direction vector — the regularized discriminant orientation in the ~20,000-dimensional gene space that best separates treated from control samples, computed as in Clark et al. Let $S_a$ be the subspace spanned by orthonormalized reference signature vectors for subtype $a$, and let $\Pi_{S_a}$ denote projection onto it. We define the margin and the cross-reactivity residue as

$$T_{a|b}(\ell) = \log \frac{\|\Pi_{S_a}\mathbf{v}_\ell\| + \epsilon}{\|\Pi_{S_b}\mathbf{v}_\ell\| + \epsilon}, \qquad \rho(\ell) = 1 - \frac{\|\Pi_{S_{\mathrm{all}}}\mathbf{v}_\ell\|}{\|\mathbf{v}_\ell\|},$$

with $\epsilon$ a small floor preventing division by zero. $T_{a|b}$ is the log-ratio analogue of the familiar binding selectivity quotient $K_{i,\text{off-target}}/K_{i,\text{target}}$, and $\rho$ is the fraction of the ligand's response that no known subtype program explains — a direct, measurable proxy for cross-reactivity.

Research Program

Three components. First, the subtype transcriptional atlas: full-transcriptome RNA-seq (not landmark panels) of cells or tissues perturbed with the best-available subtype-selective reference tools, at ≥3–5 biological replicates, which the Characteristic Direction method requires. The corpus supplies unusually clean reference pairs. The venom peptide LvIC selectively blocks the $\alpha6\beta4^*$ nicotinic receptor while [S9K]TxID blocks the $\alpha3\beta4^*$ subtype, and both were already dosed intracerebroventricularly at 10 nmol with tissue harvested — but only for proteomics, never RNA-seq, and never for both peptides in parallel (A novel α-conotoxin [D1G, ΔQ14] LvIC decreased mouse locomotor activity); that paper explicitly leaves the parallel transcriptomic comparison open. Second, BPN14770, a PDE4D-selective allosteric inhibitor, was profiled only by proteomics and only against vehicle — never against a pan-PDE4 inhibitor — so its uniquely PDE4D-driven transcriptional modules are unknown (A Novel PDE4D Inhibitor BPN14770 Reverses Scopolamine-Induced Cognitive Deficits via cAMP/SIRT1/Akt/Bcl-2 Pathway). Third, the QUESTS framework engineered CXCR4 "open" and "closed" dimer geometries that bind ligands identically yet split cleanly between $G_i$ signaling and $\beta$-arrestin recruitment, validated only with proximal biosensor assays and short-term ERK readouts (Computationally designed GPCR quaternary structures bias signaling pathway activation); running RNA-seq on these engineered states gives us reference subspaces for signaling-branch selectivity within a single receptor, which is the same mathematical object as subtype selectivity between homologs.

Second, the decomposition method: computing $T$ and $\rho$ from the atlas, with the practical care the enrichment literature demands — background universes restricted to genes actually expressed in the profiled tissue (the file documents that genome-wide backgrounds inflate housekeeping-pathway significance), and redundancy-aware aggregation of correlated subspaces, since subtype signatures will share downstream components exactly as Gene Ontology terms overlap.

Third, the coupling to selectivity prediction. The strongest chemical-only system, a one-step multiclass Extra Trees model on Mordred descriptors, classifies sigma-receptor ligands as S1R-selective, S2R-selective, non-selective, or inactive (Prediction of Activity and Selectivity Profiles of Sigma Receptor Ligands Using Machine Learning Approaches); its authors name data scarcity for selective compounds as the binding constraint. Our transcriptomic margins are precisely a way to manufacture labeled, selective-class examples at the cellular level, and we will test whether appending subspace-projection features to that architecture raises selective-compound detection. Generative approaches that optimize selectivity purely by docking against homologous pockets, such as Junction Tree VAE latent-space Bayesian optimization for FLT3 (Bayesian Optimization in the Latent Space of a Variational Autoencoder for the Generation of Selective FLT3 Inhibitors), would gain an empirical post-hoc filter: a generated candidate is only "selective" if its predicted response — via the bidirectional molecule-to-transcriptome mapping already demonstrated by BiAAE on LINCS L1000 data (Molecular Generation for Desired Transcriptome Changes With Adversarial Autoencoders) — lands in the intended subtype subspace.

Proposed Experiments

The machinery will first be sanity-checked on the benchmark the geometry was born on: the 73 transcription-factor and 130 drug perturbation experiments from GEO with ChEA/ENCODE and DrugBank silver standards used by Clark et al. (The characteristic direction); the file gives no reported number found on Lacuna for that benchmark's sensitivity values, so our replication establishes the operating point rather than comparing to one. The scientific tests then run on the three pilot systems. For the nAChR pair, we treat LvIC and TxID as ground-truth labels of subtype-of-origin and ask whether blinded RNA-seq signatures from an independent replicate cohort are correctly assigned by $T$, in caudate putamen, prefrontal cortex, and hippocampus — the three regions where LvIC's proteomics found NCS-1 and NLGN-3 changes (LvIC study). For PDE4, we compare BPN14770 against a pan-PDE4 inhibitor in the humanized PDE4D mouse (the Tyr-to-Phe allosteric-pocket humanization that species-divergence work made mandatory, Wang et al., muscarinic M4 and allosteric-site divergence, and the BPN14770 model itself), asking which transcriptional modules are uniquely PDE4D: the 325 hippocampal and 144 cortical protein changes reported there give a proteomic prior to check our RNA-seq modules against. For CXCR4, we ask whether open- and closed-dimer transcriptomes separate on the $G_i$-versus-$\beta$-arrestin subspaces — the first genome-scale readout of an engineered bias switch, filling the gap QUESTS itself names. Primary metrics: blinded classification AUC of subtype-of-origin, selective-class Matthews correlation coefficient for the ML coupling, and the correlation between binding selectivity ratios and transcriptomic margins across a ligand dose series — the last being the falsifiable heart of the claim.

Baselines

The naive reference point is the field-standard pipeline: univariate differential-expression calling with a fold-change and adjusted-$p$ threshold, then Fisher's exact overrepresentation analysis against GO and KEGG, with the background restricted to expressed genes. This is what the contemporary enrichment applications in the corpus actually do (Aspirin Eugenol Ester study; resveratrol network-pharmacology study), so it defines what "doing nothing new" buys; it will be run on our identical count matrices, with identical FDR control, and beaten only if it fails to produce any subtype-discriminating terms — the documented failure mode for coordinated low-magnitude shifts.

The strongest enrichment competitor is Enrichr's Combined Score, $z \times \ln(p)$, which corrects the systematic large-category bias of Fisher's exact test across 35 gene-set libraries (Enrichr); no reported numeric accuracy is given on Lacuna for that ranking evaluation, so we compare on our own blinded task, feeding Enrichr both the thresholded DEG lists and full ranked lists to give it its best shot. Enrichr is the strongest because it is the only enrichment system in the corpus with a demonstrated, benchmarked scoring improvement over Fisher ORA on perturbation data; we beat it if subtype-resolved subspaces achieve higher blinded classification AUC, expecting Enrichr to fail specifically because its libraries merge isoforms into generic pathway categories — a limitation its own documentation of library construction supports.

The strongest overall competitor for the selectivity-prediction question is the Lombardo Extra Trees system, which reports on its external sigma-receptor validation set an S2R ROC-AUC of 0.964 (CI 0.93–0.99) and MCC of 0.89, and an S1R accuracy of 0.936 but MCC of only 0.733, with 94 of 95 nonselective yet just 4 of 8 selective compounds correctly predicted (Lombardo et al.). We will run that exact architecture and descriptor set on our pilot ligand series with matched train/test splits and the same tuning budget, then add subspace features. Beating it means raising selective-class MCC and selective-compound detection while nonselective accuracy does not degrade — attacking precisely its reported weak point, not its strengths.

Finally, the key ablation isolates the proposal's central ingredient: the Characteristic Direction principal-angle machinery run with generic GO/KEGG/Enrichr gene sets instead of subtype-resolved subspaces. If generic subspaces separate subtypes as well as subtype-resolved ones, the contribution collapses to an application note about existing tools, and we will say so.

Why not just…?

Why not just measure binding affinity and signaling in standard in vitro assays? Because the corpus shows binding behavior and downstream consequence come apart: QUESTS open and closed CXCR4 dimers bind ligands similarly but split completely between $G_i$ activation and $\beta$-arrestin recruitment (QUESTS), and LvIC's $\alpha6\beta4^*$ blockade was known in vitro long before anyone knew it selectively depresses NCS-1 and NLGN-3 to reduce locomotor drive while sparing memory and affect (LvIC study). We are honest about the complementarity, though: tube assays remain the right tool for pocket-level triage, and our binding-versus-transcriptomic correlation experiment is designed to detect the case where the two measures are redundant for prioritization — if margins correlate tightly across a ligand series, the transcriptomic measure adds screening cost without decision value, and we would report that as a boundary condition.

Why not just use LINCS L1000 or targeted RT-qPCR panels, which are far cheaper? L1000 measures 978 landmark genes by 3'-hybridization and misses splicing and processing events that RNA-seq captures and that the corpus shows carry subtype-relevant biology, such as NAT10-regulated ac4C acetylation in tubular epithelial senescence (NAT10 study); and multi-modal datasets built on L1000 suffer severe missingness — 75.33% of compounds in the ChEMBL2K benchmark lack expression profiles (MINER). But the cost objection is real, so we build the cheap fallback into the proposal: once the atlas defines the subspaces, we test whether a distilled ~200-gene subtype panel derived from the subspace basis reproduces $T$ and $\rho$ at qPCR cost. If it does, the expensive instrument is needed only once, to build the atlas.

Why not just run DESeq2 plus GO overrepresentation, the gold standard? The file's evidence is direct: univariate methods analyze genes in isolation and miss coordinated low-magnitude shifts that the multivariate Characteristic Direction recovers across 73 TF and 130 drug perturbation experiments (Characteristic Direction), and Fisher's exact test is biased toward large gene-set categories, which Enrichr's own developers demonstrated when their rank-deviation score outperformed raw Fisher $p$-values (Enrichr). Subtype pharmacology is the worst case for both failures at once, since cross-talk between sibling subtypes manifests as subtle shifts across tens of co-regulated genes. The head-to-head run in our pilot data settles this empirically rather than by argument.

Why not just do it all computationally — docking and machine learning on chemical structures? Because the chemical-only systems fail exactly where it matters: the Extra Trees model catches 94/95 nonselective but only 4/8 selective sigma ligands (Lombardo et al.), and latent-space generative designs optimize docking selectivity scores for molecules with no documented synthetic routes and no biological validation (FLT3 Bayesian optimization), while static-pocket docking cannot see the dynamic intracellular-loop and quaternary-geometry switches that decide which pathway a receptor engages (galanin-receptor ICL2 study; QUESTS). We do not propose replacing in-silico screening — it remains the only million-compound-scalable layer — but rather supplying it with the selective-class labels it starves for, which is the documented bottleneck rather than a presumed one.

Why not just run the existing Characteristic Direction and Enrichr pipelines on new subtype RNA-seq data? This is the sharpest threat to novelty, and it is precisely the key ablation above. Generic libraries merge receptor isoforms into common pathway categories — Enrichr's own construction offers no subtype resolution — so we expect the generic-subspace run to collapse sibling-subtype signatures onto the same terms. But we treat this as an open empirical question, not a foregone conclusion: if generic KEGG subspaces separate LvIC from TxID as well as our atlas subspaces do, the subtype-resolved library is unnecessary and the honest conclusion is that this is an application of existing tools.

Decisive Checks

Four results decide the proposal. Blinded classification of subtype-of-origin across the three pilot systems must be well above chance, with replicates held out. The generic-versus-subtype ablation must show a real gap in separation. The binding-margin versus transcriptomic-margin correlation must be moderate, not near-perfect — near-perfect correlation would make the measure redundant, near-zero would make it uninterpretable, and the informative regime is in between. And the chemical-plus-transcriptomic model must recover more than 4 of 8 selective compounds at matched nonselective accuracy, converting the corpus's most clearly documented predictive failure into evidence that pathway-space selectivity is worth measuring.

References

  1. Structural insights into the ligand binding and Gi coupling of serotonin receptor 5-HT5A
  2. Computationally designed GPCR quaternary structures bias signaling pathway activation
  3. The characteristic direction: a geometrical approach to identify differentially expressed genes
  4. A novel α-conotoxin [D1G, ΔQ14] LvIC decreased mouse locomotor activity
  5. A Novel PDE4D Inhibitor BPN14770 Reverses Scopolamine-Induced Cognitive Deficits via cAMP/SIRT1/Akt/Bcl-2 Pathway
  6. Prediction of Activity and Selectivity Profiles of Sigma Receptor Ligands Using Machine Learning Approaches
  7. Bayesian Optimization in the Latent Space of a Variational Autoencoder for the Generation of Selective FLT3 Inhibitors
  8. Molecular Generation for Desired Transcriptome Changes With Adversarial Autoencoders
  9. The structural study of mutation-induced inactivation of human muscarinic receptor M4
  10. Aspirin Eugenol Ester Alleviates Gastric Injury by Inhibiting Ferroptosis and Oxidative Stress
  11. Multi-target modulation of the homologous recombination pathway by resveratrol promotes DNA damage repair in POI: integrated network pharmacology, molecular dynamics simulation, and experimental validation
  12. Enrichr: interactive and collaborative HTML5 gene list enrichment analysis tool
  13. NAT10 Promotes Tubular Epithelial Cell Senescence in Cisplatin-Induced Acute Kidney Injury by Regulating DDX17
  14. Multi-modal Contrastive Learning with Negative Sampling Calibration for Phenotypic Drug Discovery
  15. Molecular basis for allosteric agonism and G protein subtype selectivity of galanin receptors