Spatial transcriptomics, which measures gene expression at fixed spots across a tissue slice, gives every spot a mixed signal. Each spot averages several cells, so a regulatory relationship that exists only in a rare immune cell sitting among tumor cells disappears into the average. Pathologists looking at the matching stained tissue image (an H&E slide) can often see which cells are in a spot. In the regions where seeing is hardest, they disagree with each other. Those borderline zones are exactly where cell states blur into each other, which is also where gene-network inference from mixed signals fails.
This note proposes a model that estimates two things together, at genome-relevant scale: which cell types make up each spot, and a separate sparse regulatory network for each cell type. The model's prior on spot composition comes from a morphology classifier trained on the full spread of several pathologists' labels rather than on a majority vote. Where pathologists agree, the image pins down the mixture and the model can spend its statistical budget on network edges. Where they disagree, the prior flattens automatically and the transcriptome decides. The claim is that measured human disagreement is the right way to set how much an image prior should be trusted. We also claim that this prior, used inside joint unmixing-plus-network inference, recovers cell-type-specific regulatory edges in minority cell types that two-step pipelines and image-free joint models miss.
Where this sits relative to existing work
Three lines of work each solve part of the problem.
The first line is joint unmixing and network inference. VMPLN showed that clustering cells first and inferring networks second gives wrong topologies when cell types overlap. Its remedy is to cluster and estimate per-type precision matrices together under a Poisson log-normal model, solved with ADMM, which splits the optimization into smaller pieces that can run in parallel. This paper supplies our core mixture-of-networks model and motivates joint inference. Its prior over cluster membership, however, is uninformed and it works only on dissociated cells. PMF-GRN shows that variational inference with minibatch stochastic gradients on GPUs can scale network inference and return usable edge uncertainty. We reuse its optimization strategy. It neither unmixes nor uses images.
The second line is image-informed deconvolution. The CTA pipeline classifies single nuclei on H&E into tumor, immune and stroma and maps them onto Visium spots, but only as hard labels used to check deconvolution tools. Count Bridges deconvolves spots into integer single-cell counts. Neither infers networks, and both treat genes as unconnected during unmixing.
The third line is soft labels from multiple observers. GleasonXAI trained on soft labels from 54 pathologists and reached Dice 0.713 ± 0.003, against 0.691 ± 0.010 for direct segmentation. That supports the view that disagreement carries signal rather than noise, but the work is image-only.
The proposal combines three things none of these papers has together: soft morphological composition priors, per-type sparse networks, and joint inference over spots.
Core Claim
Each spot $s$ has a nucleus count $n_s$ from segmentation. A morphology classifier gives each spot soft type proportions $\pi_s$ over $K$ cell types, trained on multi-pathologist label distributions. The model treats the spot's latent composition $w_s$ as drawn from
$$ w_s \sim \mathrm{Dirichlet}(\alpha_0 + \lambda\, n_s\, \pi_s). $$
Here $\alpha_0$ is a weak baseline concentration, and $\lambda$ is a learned global scale on how much the image is trusted. The prior's sharpness therefore depends on how confident the soft classifier is, and that confidence was learned from how much pathologists agreed.
Observed counts follow
$$ y_{sg} \sim \mathrm{Poisson}\Big(\ell_s \sum_k w_{sk}\, \exp(x_{skg})\Big), \qquad x_{sk\cdot} \sim \mathcal{N}(\mu_k, \Theta_k^{-1}), $$
where:
- $y_{sg}$ is the count of gene $g$ in spot $s$;
- $\ell_s$ is library size;
- $x_{sk\cdot}$ is the latent log-expression of type $k$ in spot $s$;
- $\Theta_k$ is a sparse precision matrix, whose nonzero entries are the type-$k$ network.
Inference is mean-field variational with minibatches of spots, in the PMF-GRN style. A graphical-lasso penalty on each $\Theta_k$ handles sparsity. The cost per step is linear in batch size times genes times $K$, plus a per-type sparse precision update.
The analytical claim can be falsified. Spots where pathologists disagree more should show higher posterior composition entropy and lower edge confidence. If disagreement shows no relation to molecular mixing, the motivation for disagreement-calibrated priors collapses, and we will report that.
Research Program
The work has three parts.
The first part is data, and it is the honest bottleneck. No dataset in the file pairs multi-observer labels with spatial transcriptomics. We will collect them on the CTA cohort's breast tumor H&E images (23 Visium sections, 4 patients) and on its HER2-positive Xenium sample. Six to eight pathologists will label nuclei in sampled fields as tumor, immune or stroma, with an "uncertain" option. That label set matches CTA's classes and keeps annotation time modest.
The Xenium sample is central because Xenium measures expression at near single-cell resolution. We can aggregate its cells into Visium-sized pseudo-spots of known composition. A reference network per cell type can then be estimated from the true per-cell Xenium data and serve as a proxy ground truth for what a spot-level method should recover. One limitation must be stated: Xenium measures a targeted gene panel, so network evaluation is restricted to that panel.
The second part is the morphology prior. We will train the nucleus classifier with soft-label loss on the pathologist distributions, following GleasonXAI. A version trained on majority-vote labels will serve as an ablation.
The third part is scale and robustness on data with true networks. BEELINE synthetic trajectories from the PMF-GRN suite and the Kang and Zheng PBMC mixtures from VMPLN have no images. On these we simulate pseudo-spots and synthetic morphology priors at controlled noise levels. This shows how much prior accuracy the method needs before it helps, and it measures wall-clock scaling against VMPLN's ADMM solver as the numbers of genes and spots grow.
Baselines
The naive reference fits a single network to all spots, pooling them as if each spot were one cell type. Any per-type method must beat it on minority-type edges, or it has learned nothing about heterogeneity.
The strongest practical competitor is the standard two-step pipeline. It deconvolves spots with RCTD or Cell2location using the breast scRNA reference supplied with the CTA cohort, fixes those proportions, and then fits per-type graphical lasso or GENIE3 on the reconstructed per-type expression. The same pipeline will also be run with CTA's hard morphology labels in place of RCTD or Cell2location. This is what a careful analyst would do today. It will receive the same gene set, the same scRNA reference, the same GPU budget, and an equal hyperparameter search budget, with sparsity chosen by the same held-out likelihood criterion.
The strongest existing joint method is VMPLN, applied to spot counts with its uninformed membership prior. For context, PMF-GRN will be run on the dissociated scRNA reference to give a non-spatial network.
On these benchmarks the file has no reported numbers: no reported result found on Lacuna for VMPLN's pAUPRC values, for PMF-GRN's AUPRC values, or for numeric deconvolution correlations in the CTA paper. We will therefore rerun every baseline ourselves rather than quote figures.
Two ablations isolate the key ingredients. The first replaces soft priors with hard majority-vote priors. The second removes the network term, setting each $\Theta_k$ to diagonal, so the model becomes pure deconvolution; this tests whether network structure also improves unmixing.
Beating the baselines means higher early precision on the Xenium proxy networks for the immune type, the minority class in these tumors, with confidence intervals from bootstrap over pseudo-spot draws that exclude the best baseline. Overall proportion recovery must not be worse.
Why not just…?
Why not just deconvolve with RCTD or Cell2location and then run per-type regression? This is the real threat. If deconvolved proportions were accurate enough, fixing them and fitting a weighted regression would match a joint model at lower cost. VMPLN's simulations show two-step pipelines failing under strong mixing, but on dissociated cells, not on spots. The proposal therefore makes this comparison the primary test, reported separately for spots stratified by pathologist disagreement. If the two-step pipeline matches the joint model even in high-disagreement spots, we will conclude the joint machinery is unnecessary and say so.
Why not just run VMPLN on the spots, with no images? The transcriptome alone may suffice when types differ strongly. The synthetic-prior experiment answers this directly by sweeping prior noise. If VMPLN equals the method once prior accuracy drops to the level the real classifier actually achieves, the image contributes nothing.
Why not just use majority-vote morphology labels? GleasonXAI found soft labels improved Dice from 0.691 to 0.713. That gain is modest and was measured on an image task, so it does not establish a benefit for network inference. The hard-prior ablation settles this, with the effect expected only in high-disagreement spots.
Why not just predict the clinical label end-to-end from the slide? Shamai et al. reach AUC 0.95 for ER from H&E, so prediction alone is not the gap. The goal here is regulatory structure per cell type, which an end-to-end classifier does not produce.
Decisive Checks
Three results decide the proposal.
The first is whether, on Xenium pseudo-spots, the morphology-guided joint model improves immune-type early precision over both the two-step pipeline and VMPLN, with the gain concentrated in high-disagreement spots.
The second is whether pathologist disagreement correlates with posterior composition entropy across spots, measured by rank correlation with a permutation null. A null result would mean disagreement reflects visual difficulty unrelated to molecular mixing.
The third is whether the method's runtime grows roughly linearly in spots on BEELINE-scale and PBMC-scale pseudo-spot sets, while VMPLN's grows faster.
If the first check fails against the two-step pipeline, the contribution shrinks to a disagreement-annotated spatial benchmark plus a negative result. That is still useful, but it is a different paper.
References
- Single-cell gene regulatory network analysis for mixed cell populations
- PMF-GRN: a variational inference approach to single-cell gene regulatory network inference using probabilistic matrix factorization
- Computational pathology annotation enhances the resolution and interpretation of breast cancer spatial transcriptomics data
- Count Bridges enable Modeling and Deconvolving Transcriptomic Data
- Pathologist-like explainable AI for interpretable Gleason grading in prostate cancer
- Clinical utility of receptor status prediction in breast cancer and misdiagnosis identification using deep learning on hematoxylin and eosin-stained slides