Telling Cell-Type Partitioning from Regulatory Divergence in Duplicated Genes Without a Single-Cell Atlas

When a genome duplicates a gene, the two copies often split up the work. One copy may be switched on in one cell type and the other copy in another, or both may stay in the same cells but respond to different signals. Most studies of expanded gene families in non-model species rely on bulk tissue RNA-seq, meaning expression measured from ground-up tissue that mixes many cell types, and on co-expression networks built from it. In that data the two scenarios look the same. Two copies land in different co-expression modules whenever the proportions of cell types shift from sample to sample. They land in the same module whenever they happen to be expressed by the same infiltrating cell type. Resolving this normally requires a single-cell atlas of the species, which non-model organisms rarely have.

We propose a way around this. First, estimate each sample's cell-type proportions using only genes with clean one-to-one counterparts in a well-studied proxy species, so the proxy's cell-type reference is used only where it is trustworthy. Then treat the duplicated genes, which cannot be mapped cleanly, as held-out responses. For each copy, a regression on those proportions estimates which cell types express it. Whatever co-expression is left among copies after composition is removed measures regulatory coupling that happens within cells. The claim is that separating the genes used to estimate composition from the genes being explained turns the main weakness of proxy references into a working design. The result is two things per duplicated gene: an estimated cell-type expression profile, and a test that says whether a module split between duplicates reflects cell-type partitioning or regulatory divergence.

Why this gap exists

Four literatures each handle one piece. Deconvolution, meaning the computational unmixing of bulk profiles into cell-type fractions, is known to suffer misattribution: signal from cells missing in the reference gets pushed into the closest available category (Bergenstråhle et al. (2020)). Proxy references add systematic skew because orthologous genes diverge in expression (Tello‐Palencia et al. (2024)). The closest prior work is the rat blood study of Tello‐Palencia et al. (2024). It deconvolved rat bulk blood with a mouse reference and used the resulting fractions as covariates before differential expression and pathway enrichment. It used one-to-one orthology throughout and never asked what the non-mapped paralogs were doing. That study motivates our design, and we reuse its covariate idea as a baseline.

Weighted gene co-expression network analysis (WGCNA) groups genes into modules by the similarity of their correlation neighborhoods and summarizes each module by its first principal component. It has been applied directly to species with expanded families. Gao et al. (2025) combined genome-wide expansion detection, WGCNA across tissues and GO/KEGG enrichment in Alpinia officinarum. The cross-species osteosarcoma study (Cross-Species Gene Expression Analysis…) built human–dog consensus modules and nominated CD86 as a hub. Neither study deconvolved its data. So neither can say whether a module reflects co-regulation within cells or a varying immune infiltrate, and the osteosarcoma study also discarded non-one-to-one genes.

Reference-free deconvolvers such as FAST (Zhang et al. (2025)), a spatially regularized non-negative matrix factorization, and Starfysh (He et al. (2024)), a variational autoencoder anchored on marker genes and histology, avoid external references. They produce unlabeled or marker-defined factors, however, and are built around spatial data. Marker-based methods such as Celloscope (Geras et al. (2023)) assume marker exclusivity, which duplication erodes. We borrow Celloscope's "dummy cell type" idea to absorb unmodeled signal. Evolutionary studies show that duplicates really do partition function, for example the divergent alpha-tubulin copies in bdelloid rotifers (Eyres et al.). What is missing is a tool that recovers this partitioning from bulk data in a species without an atlas.

Core Claim

Let $x_{gs}$ be the normalized expression of gene $g$ in bulk sample $s$ from the target species. Let $\hat p_{ks}$ be the estimated fraction of cell type $k$ in sample $s$. Crucially, $\hat p$ is computed only from one-to-one orthologs using the proxy reference, plus one residual "unassigned" component. For every duplicated gene $g$, which never enters the estimation of $\hat p$, we fit

$$x_{gs} = \sum_{k} w_{gk}\, \hat p_{ks} + \varepsilon_{gs},\qquad w_{gk} \ge 0,$$

where $w_{gk}$ is the inferred per-cell-type expression of gene $g$ and $\varepsilon_{gs}$ is the part of its expression that composition does not explain. For a pair of duplicates $(g,h)$, the drop in bulk correlation between them splits into two parts. The composition part is predicted by their fitted profiles $w_{g\cdot}$ and $w_{h\cdot}$. The regulatory part is the residual correlation $\mathrm{cor}(\varepsilon_{g\cdot}, \varepsilon_{h\cdot})$. We call a split "cell-type partitioning" when the fitted profiles point to different cell types and the residual correlation is indistinguishable from the one-to-one-ortholog null. We call it "regulatory divergence" when the profiles overlap and the residuals are decorrelated. We then run WGCNA on the residual matrix to obtain modules that are, as far as bulk data allows, cell-intrinsic.

Holding out duplicated genes does two things. It stops them from distorting $\hat p$, since their cross-species signatures are the least reliable. It also removes the circularity of explaining genes with proportions they helped estimate. The residual component catches cell types missing from the proxy reference. When a duplicated gene loads heavily on that component, this is itself a finding: the gene may mark a cell state absent from the reference.

Research Program

The method is built in three layers. The first is composition estimation on one-to-one orthologs. We use a constrained non-negative least-squares deconvolution against the proxy reference, with an added free residual component. A variant uses a pretrained-then-adapted autoencoder in the style of TAPE (Chen et al. (2022)), so we can check that conclusions do not hinge on the deconvolver. The second layer is per-paralog profile estimation with non-negative regression. Uncertainty comes from bootstrapping samples, because bulk cohorts are small and cell-type proportions are often correlated with one another. The third layer is the pairwise decomposition test and residual WGCNA.

Before any biological claim, the program has to establish two things. The first is that the method is identifiable: how many samples, and how much independent variation in proportions, are needed before $w_{gk}$ is recoverable. We characterize this through the condition number of the $\hat p$ matrix and report it for every real cohort. Cohorts that fail the check are reported as uninformative rather than analyzed. The second is that the test is calibrated, meaning its false-call rate is controlled when the truth is known.

Functional enrichment enters as a constraint, not as a contribution. Duplicates inherit near-identical annotations, and gene-level overrepresentation tests then count one family many times (Fan et al. (2025) describes an 11-member PR-1 expansion of the kind that would dominate a gene-level test). All enrichment is therefore reported with families collapsed to one unit, alongside the conventional gene-level result. Enrichment then serves as a readout. Modules removed by composition adjustment should be enriched for cell-identity terms, and residual modules should retain process terms.

Baselines

The naive reference point is standard bulk WGCNA on all genes, as run by Gao et al. (2025) and the osteosarcoma study. Paralog module membership is read off directly, and each pair counts as "diverged" if the two genes fall in different modules. It is the default practice, and any method that cannot beat it at telling partitioning from regulatory divergence has no reason to exist.

The strongest existing competitor is composition-covariate adjustment in the manner of Tello‐Palencia et al. (2024). Proportions are estimated from the proxy reference using all mappable genes, including paralogs forced to a single ortholog. These proportions are regressed out of every gene before WGCNA, and each paralog's cell type is assigned by its correlation with the estimated fractions. This is the most direct existing way to address composition confounding, and it differs from our method in only two respects: which genes estimate composition, and whether per-paralog profiles and a formal decomposition are produced. Both methods get the same deconvolution solver, the same samples, the same WGCNA soft-threshold selection rule and the same bootstrap budget. On deconvolution accuracy, the file's reported numbers are all on different tasks. FAST reports Pearson correlation of 0.93 between true and estimated proportions on a simulated mouse nervous-system spatial benchmark. Starfysh reports RMSE ≈ 0.09 on spot unmixing. Celloscope reports errors between 0.01 and 0.03 on 60 synthetic spatial datasets. No reported result found on Lacuna for proxy-reference bulk deconvolution on Kassandra or 3CA pseudo-bulk, and none for paralog cell-type assignment, so we will establish the baseline numbers ourselves. Beating it means a statistically significant improvement in paralog cell-type assignment accuracy and in partitioning-versus-divergence classification on held-out simulations, without worse proportion accuracy.

A reference-free comparison runs plain non-negative matrix factorization on all genes, the non-spatial core of FAST, with factors matched to cell types by marker overlap. This tests whether an external reference is needed at all.

Two ablations isolate the key ingredient. In the first, paralogs are included in the composition estimation, which tests whether holding them out matters. In the second, paralog counts are summed into a family total, which tests whether per-copy resolution matters.

Why not just…?

Why not just regress out estimated proportions, as the rat blood study did? This is the real threat. If paralogs barely affect the estimated proportions, holding them out changes little. The remaining contribution would then be the explicit per-paralog profiles and the calibrated test, which covariate adjustment does not produce but a careful analyst could approximate. The first ablation settles this directly. We report the gain from holding out paralogs as a function of the fraction of signature genes that are duplicated. If the gain is negligible everywhere, we will say so and present the method as a formalization of covariate adjustment with uncertainty quantification.

Why not just restrict to one-to-one orthologs, or sum each family? Restricting to one-to-one orthologs removes exactly the genes under study. Expansions linked to adaptation, such as trehalose synthesis in high-altitude ciliates (Wang et al. (2026)), are by construction not one-to-one. Summing a family erases the cell-type contrast between copies that sub-functionalization creates. The family-sum ablation measures how much assignment accuracy this costs.

Why not just use reference-free factorization and skip the proxy species? Reference-free factors come without labels. Highly similar, correlated duplicates are also prone to being split arbitrarily across factors, and FAST (Zhang et al. (2025)) depends on spatial and histology regularization that bulk cohorts lack. The NMF baseline shows whether this matters in practice.

Why not just collapse families in enrichment and stop there? For enrichment inflation alone, this simple fix probably suffices, which is why we adopt it rather than propose a new test. It does not address the composition question, which is the core of this proposal.

Proposed Experiments

The decisive experiment is a semi-synthetic benchmark where the truth is known. From the 3CA single-cell atlas (over 2.7 million cells across 15 cancer types, used by POPPY), we build pseudo-bulk cohorts with controlled, varying cell proportions. Human duplicated genes are identified from family annotations. The true per-cell-type profile of each duplicate comes from the atlas itself. The reference is deliberately degraded in three ways: restricted to one-to-one orthologs of a model species, with one cell type removed, and with expression perturbed to mimic cross-species divergence. We then plant two kinds of pairs. Some are "partitioned" pairs whose copies come from different cell types. Others are "regulatory" pairs whose within-cell-type expression is decoupled by an injected sample-level program. Metrics are the correlation between inferred and true $w_{g\cdot}$, top-cell-type agreement, and area under the ROC curve (the curve of true versus false positive rate) for classifying partitioning versus divergence. We also track calibration of the residual test on one-to-one null pairs. All of these are tracked as sample size and proportion variability change. Kassandra's 2,286 bulk profiles simulated from physically sorted cells provide a second test of proportion recovery on real sorted inputs.

The second experiment is a biological reanalysis. We revisit the human and canine osteosarcoma cohorts (154 and 117 samples) and the GSE21257 validation set, where module C3 was linked to metastasis-free survival (p = 0.021) and CD86 was nominated as a hub (p = 0.0017). The question is whether C3 survives composition adjustment. If it does not, it was an immune-infiltration signal. If it does, its residual version should still associate with survival. Duplicated genes excluded by the original study are placed back into residual modules. The rat whole-blood data (GSE278528) with a mouse proxy provides a second application, testing whether interferon-family paralogs split by cell type or by regulation.

The results would be negative for the idea in either of two cases. The held-out design might fail to beat covariate adjustment in the semi-synthetic benchmark. Alternatively, identifiability might fail at the cohort sizes typical of non-model species, as in Alpinia-scale multi-tissue panels. In the second case, the honest conclusion is a sample-size threshold below which bulk data cannot answer the partitioning question.