Chromatin Competence Priors for Spatial Differential Expression

Spatial transcriptomics measures which genes are active at each location in a tissue slice. A common use is finding genes that a given cell type switches on in one tissue region and not in another, for example endothelial cells inside a tumor versus outside it. These comparisons produce many false hits. Cell boundaries drawn on the image are imperfect, and RNA from a neighboring cell leaks into the target cell's profile, so a macrophage gene can appear "upregulated in endothelial cells" simply because endothelial cells near tumors sit next to macrophages. We propose to catch this leakage with a different kind of evidence: whether the target cell type's DNA is physically open at that gene. Chromatin accessibility profiling (ATAC-seq, which measures where DNA is unwrapped and reachable by regulatory proteins) tells us which genes a cell type is structurally able to transcribe. A cell type whose DNA is tightly closed at a gene's promoter and enhancers is very unlikely to be the true source of that gene's transcripts.

The obvious simpler check is to ask a dissociated single-cell RNA atlas whether the cell type ever expresses the gene. That check has a blind spot, and the blind spot is exactly the discovery we care about. Genes induced by a tissue niche (hypoxia, contact with tumor cells, local signaling) are often silent in dissociated reference cells, which have been removed from that niche. An RNA-based filter therefore discards true niche-induced genes along with the contamination. Chromatin is primed before transcription begins, so it can separate "never possible in this cell type" from "possible but currently off." Our claim is that an accessibility-derived competence prior keeps precision against neighbor contamination while recovering niche-induced genes that RNA-reference filtering throws away.

Core Claim and Position Relative to Prior Work

The closest work is 2DE. It flags a candidate gene as spurious when the target cell group's spatial neighbors express it more strongly than the target group itself, and it scores confidence with a Gaussian process classifier layered on standard tests. On 10X Xenium breast cancer, 2DE raised precision from 0.47 to 0.88 with a t-test and from 0.41 to 0.47 with latent-variable DE. 2DE uses only RNA. It has a known failure mode: a gene that the target cells truly induce in a niche where neighbors also express it gets flagged as contamination. It is also scored against a dissociated scRNA-seq reference, which by construction cannot reward niche-induced genes. POLARIS separates composition changes from region-specific expression in spot-based data. It assumes the regional shift is shared by all cell types in a region, and it uses histology rather than any regulatory evidence. Integration methods such as scBridge and SCARF align dissociated RNA and ATAC profiles but never touch spatial coordinates or differential testing. The human early skeletal development atlas collected scRNA-seq, scATAC-seq and spatial data, but it analyzed them in parallel.

Two strands of evidence shape how we use chromatin data. Accessibility is necessary but not sufficient for transcription: primed elements stay open without driving expression (Oncogenic ERRB2 signals through AP-1; Chromatin Accessibility Impacts Transcriptional Reprogramming in Oocytes). Accessibility also tracks transcriptional fidelity (Chromatin accessibility associates with protein-RNA correlation in human cancer). Together these argue for a one-sided use. Closed chromatin should lower belief that a cell type produces a gene. Open chromatin should not raise belief that it is currently expressed. Our model enforces this asymmetry.

Research Program

We will build a spatial count model in which each segmented cell's observed transcripts are a mixture of its own expression and leakage from adjacent cells. The cell's own expression is gated by a competence variable whose prior comes from chromatin accessibility. For cell $i$ of type $t$ in niche $n$, the count of gene $g$ is

$$y_{ig} \sim \mathrm{NB}\Big(s_i\big[z_{tg}\,\mu_{tgn} + \textstyle\sum_{j} a_{ij}\,\hat{c}_{jg}\big],\ \phi_g\Big), \qquad z_{tg} \sim \mathrm{Bernoulli}(\pi_{tg}).$$

Here $s_i$ is the cell's size factor. $\mu_{tgn}$ is the niche-specific expression of gene $g$ in type $t$. $a_{ij}$ is a learned leakage weight that decays with the distance between cell $i$ and neighbor $j$. $\hat{c}_{jg}$ is the neighbor's estimated own expression. $\phi_g$ is the per-gene overdispersion, estimated with the empirical Bayes shrinkage used by edgeR/DESeq2-style methods. $z_{tg}$ indicates whether type $t$ is competent to transcribe $g$. The prior $\pi_{tg}$ is a calibrated function of accessibility at the gene's promoter and at distal peaks linked to it in type-matched scATAC-seq, with a floor so that closed chromatin reduces belief but never removes it. Differential expression between niches is tested on $\mu_{tgn}$, and each call is reported together with the posterior probability that $z_{tg}=1$. A gene counts as a cell-intrinsic niche DEG only if both are high.

Three pieces of work follow. First, a calibration study learns the map from accessibility to $\pi_{tg}$ on genes whose cell-type specificity is uncontested. The consensus-peak reference of A generic reference defined by consensus peaks for scATAC-seq gives a common peak coordinate system across datasets. Second, the model is fit with variational inference, using the same latent-variable machinery that 2DE's lvm-DE baseline uses. Third, we will build a new evaluation set that does not inherit the scRNA-reference bias described below.

Data and Evaluation Design

The primary tissue is the 10X Xenium human breast cancer dataset with its scRNA-seq reference, so that results are directly comparable to 2DE. Breast cancer chromatin profiles by cell type will come from CATA, with ATACdb for normal stromal and immune types. This accessibility data is not from the same patients, which is a real limitation. We therefore add the skeletal development atlas as a second setting where scRNA, scATAC and spatial data come from matched tissue.

2DE's precision is measured against the dissociated scRNA reference, and that reference cannot credit niche-induced genes. We will therefore score methods in three ways. The first is the reference-based precision used by 2DE, for comparability. The second is a semi-synthetic benchmark. We take well-segmented Xenium cells, inject leakage from real neighbors at controlled rates, and plant niche-induced genes: genes that are expressed in the target type only in one niche and are absent from the dissociated reference. This gives known ground truth for both error types. The third is a contamination-resistant silver standard built from transcripts that fall inside the DAPI-stained nucleus. Nuclear-restricted counts are much less exposed to leakage from neighboring cells, so niche DEGs that replicate in nuclear-only counts are more trustworthy. We report precision and recall on contamination-type false positives and on niche-induced true positives separately.

Baselines

The naive reference point is a per-cell-type t-test between niches on segmented Xenium cells, run with no contamination handling. The file reports its precision as 0.47 under Proseg segmentation (2DE). We also run latent-variable DE through scvi-tools, reported at 0.41.

The strongest existing competitor is 2DE applied on top of the t-test. The file reports a precision of 0.88 on Xenium breast cancer under Proseg segmentation. It is the strongest because it attacks exactly our target error, neighbor contamination, and it needs no extra data modality. We will run the released 2DE on the same segmentation, the same cell-type labels and the same niche definitions, and we will match our model's hyperparameter search budget to the tuning 2DE receives. Beating 2DE means matching its reference-based precision within noise while achieving clearly higher recall on planted niche-induced genes in the semi-synthetic benchmark and on nuclear-replicated genes. A precision gain alone is not the target. We also report 2DE combined with our competence posterior, since the two methods may stack.

The key ablation isolates the source of the competence prior. We keep the model fixed and set $\pi_{tg}$ in four ways: uniformly, from scRNA reference pseudobulk expression, from scATAC accessibility, and from accessibility with peaks shuffled across genes within each cell type. The shuffled version tests whether accessibility carries real gene-level information. A variant with no leakage term tests whether the mixture structure matters apart from the prior.

Why not just…?

Why not just filter candidate DEGs by whether the cell type expresses the gene in the scRNA-seq reference? This is the most serious threat. On contamination alone, it may match or beat chromatin, because the reference is a direct RNA measurement. Its weakness is structural: dissociated cells have lost their niche, so genes induced by the niche look absent and get filtered out. The ablation that compares an RNA-derived prior with an accessibility-derived prior, stratified by planted niche-induced genes versus contamination genes, settles this. If the RNA prior is as good on niche-induced genes, our claim fails and we will say so.

Why not just use 2DE? 2DE already reaches 0.88 precision. It also rejects any gene where the neighbors express more than the target, which includes true co-induction in shared niches. Competence priors can rescue such genes when the target's chromatin is open, and they can reject contamination even when the neighbors' expression happens to be weak. The stacked comparison will show whether the two signals are complementary or redundant.

Why not just use histology, as Starfysh and POLARIS do? Starfysh and POLARIS use morphology to estimate composition. Morphology carries no gene-specific information about which loci a cell type can transcribe, and POLARIS's shared regional shift explicitly gives up cell-type-specific responses. Histology could inform the leakage weights $a_{ij}$ through cell density, and we treat that as an optional input rather than a competitor.

Why not run Robust PCA directly on the spatial matrix? The RPCA DEG method needs a sample count of at least three times the matrix rank for its reported accuracy of over 90%. Its sparse component also cannot distinguish leakage from true activation. We include it only as a secondary reference on the semi-synthetic benchmark.

Decisive Checks

The proposal succeeds if three things hold. On the semi-synthetic benchmark, the accessibility prior keeps contamination precision close to 2DE's while recovering substantially more planted niche-induced genes than either 2DE or the RNA-reference prior. Shuffling the accessibility peaks erases that gain. Niche DEGs rescued by the competence prior replicate in nucleus-only counts at a rate close to the rate for DEGs all methods agree on. The proposal fails if the gain appears only in matched-tissue data and disappears with CATA's unmatched profiles, which would limit the method to multiome-profiled tissues. A secondary extension to Visium-style spots, using POLARIS's HER2+ breast cancer data, would replace segmented cells with inferred composition and would be pursued only if the primary checks pass.