Niche-Attributable Pathway Enrichment for Glia in Alzheimer's Tissue

When a microglial cell, one of the brain's resident immune cells, switches on an inflammatory program in Alzheimer's disease, is it responding to something inside itself or to what surrounds it: nearby astrocytes, dying neurons, an amyloid plaque? Current single-cell tools cannot answer this. Pathway enrichment analysis groups genes into curated functional sets such as "NF-κB signaling" and asks which sets change with disease. It treats each dissociated cell as if it lived in a vacuum. We propose a different enrichment test. For each curated pathway and each cell type, it estimates how much of the pathway's activity is explained by the cell's spatial neighborhood beyond the cell's own identity. The test corrects for the main artifact that fakes such signals, which is transcripts from neighbors leaking into a cell's measurement. It is calibrated against permutations that keep local cell composition fixed.

The output is a set of curated pathways split into two parts: the genes set by the neighborhood, and the genes that behave the same regardless of surroundings. These context-labeled gene sets can then be scored in the large dissociated brain atlases, which hold millions of nuclei but have no spatial coordinates. That gives those atlases a principled readout of environmental effects they cannot measure directly. The bet is that separating neighborhood-driven signal from cell-intrinsic signal will expose disease-linked glial pathway changes that cell-type-averaged enrichment misses.

Why this, and why now

The motivating biology is well documented in the file. Work in human stem-cell-derived co-cultures shows that motor neuron death in ALS is driven by astrocytes turning toxic, not by the neurons alone (Phenotyping Neurodegeneration in Human iPSCs; Wei Zheng et al.). Chun et al. review a self-reinforcing loop between microglia and astrocytes in Alzheimer's disease. They report that the spatial extent of glial activation predicts cognitive decline better than plaque density does. Both papers motivate the proposal: they say the relevant variable is context. Neither gives a way to measure context-dependence of specific pathways from transcriptomes.

The analogy to the "vacuum bias" in molecular modeling is more than rhetorical. Shee et al. show that drug tautomer preferences invert between vacuum and water. TerraBind explicitly names the missing cellular context as a limitation of pairwise binding models. The transcriptomic version of the same error is scoring pathways on isolated cells and then reading the result as biology that may be environmental. We borrow one concrete tool from that line of work. MeToken defines a micro-environment around a protein residue as the union of a fixed-radius sphere and the K nearest spatial neighbors. We use the same two-part definition for cellular neighborhoods, and we treat the radius and K as hyperparameters to be tested rather than assumed.

Empirically, the gap shows up as a tension in the file. A standard re-analysis of an Alzheimer's cohort, GSE243243 (Analyzing single-cell transcriptome sequencing data reveals immunological mechanisms of microglia-associated diseases), ran differential expression followed by GO and KEGG overrepresentation and found no significant change in APOE or TREM2 in microglia. ResidPCA instead removes cell-type baselines and then looks for states that cut across cell types. It found 30 chromatin states enriched for Alzheimer's heritability, including a neuron–oligodendrocyte–microglia axis. One reading of this tension is that disease-relevant glial programs are confined to a subset of cells in particular niches, and averaging over all microglia in a donor dilutes them. Our test is designed to check that reading directly.

Core Claim

We claim that pathway enrichment for glia can be reframed as a question of attribution. The question is how much of a curated pathway's variation, within a fixed cell type, is predictable from that cell's neighborhood. This quantity can be estimated without the circularity of using the cell's own expression. It can be protected against transcript spillover and calibrated against composition-preserving spatial nulls. Pathways that are niche-attributable in spatial data will carry disease-stage signal when transferred to dissociated atlases that curated, context-free gene sets do not.

Concretely, for cell $i$ of type $c$ and gene $j$ we fit

$$ x_{ij} = a_{cj} + b_j^\top u_i + g_j\big(\mathcal{N}(i)\big) + \lambda_j \sum_{k \in \mathcal{N}(i)} w_{ik}\, x_{kj} + \varepsilon_{ij}. $$

Here $x_{ij}$ is normalized expression and $a_{cj}$ is the cell-type baseline. The vector $u_i$ holds technical and sample covariates such as library size and slide. The neighborhood $\mathcal{N}(i)$ contains the other cells within radius $R$ or among the $K$ nearest, together with the distance to the nearest plaque. The term $g_j$ is a niche encoder that never sees cell $i$'s own expression. The last term is a spillover model: the leaked signal is proportional to neighbors' expression $x_{kj}$, weighted by contact $w_{ik}$, with a gene-level leakage coefficient $\lambda_j$. We estimate $\lambda_j$ from marker genes that are biologically exclusive to the neighboring type but appear in cell $i$'s measurement.

The niche-attributable fraction of gene set $S$ in cell type $c$ is the held-out variance explained by $g$ over genes in $S$, relative to a background of matched genes. Its null distribution comes from reassigning neighborhoods among cells of the same type within the same slide and the same composition bin.

Research Program

The work has three parts.

The first part builds the estimator and its null. The niche encoder is a permutation-invariant set model over neighbor cells, in the spirit of NicheFlow's microenvironment transformer (NicheFlow), which models a niche as an attributed point cloud of the cells within a radius. Unlike NicheFlow, we do not generate niche trajectories. We use the neighborhood only as a predictor, trained with cross-fitting across slides so that variance explained is always measured out of sample. We also implement the same attribution with a linear $g$ over neighbor-type proportions and mean neighbor expression. The deep encoder stays in the method only if it beats the linear version on held-out prediction.

The second part produces context-split gene sets. For each GO or KEGG set and each glial type, we separate member genes whose variance is niche-attributable from those that are not. For niche-attributable genes, we record which neighbor features drive them, such as astrocyte reactivity, neuronal loss, or plaque distance, using attribution from the encoder.

The third part transfers the split gene sets to dissociated data. We score them with a standard per-cell scorer on the Morabito snRNA-seq cohort used by ResidPCA, on the postmortem cohorts in the scMamba compendium (scMamba), and on GSE243243. We then ask whether the niche-driven halves track disease stage and heritability better than the full curated sets or their intrinsic halves.

The spatial data requirement is the main practical risk. The file catalogs the Mouse Brain Aging spatial series used by NicheFlow and the multi-organ HEST-1k resource (HEST-1k). It does not catalog a human Alzheimer's spatial transcriptomics cohort with matched plaque staining. We would use such public cohorts where they exist and treat the aging mouse brain as a development and replication set. Any human-tissue claims depend on that data.

Baselines

The naive reference point is the pipeline of the GSE243243 re-analysis. It takes microglial differentially expressed genes between disease and control and runs hypergeometric overrepresentation against GO and KEGG. We run it unchanged on the same spatial cells, ignoring coordinates. It fixes the floor. On real data, beating it means recovering disease-stage associations it misses, and replicating them on held-out donors.

The simplest serious competitor is per-cell pathway scoring with AUCell, followed by a linear mixed model that regresses the score on neighbor-type proportions and plaque distance, with slide as a random effect. We expect this to be the strongest competitor, because it asks nearly our question with off-the-shelf parts. It lacks only the spillover term, the composition-preserving null, and the gene-level split. It gets identical neighborhoods, identical covariates, and the same held-out-slide protocol. We beat it only if its calls fail calibration or specificity in the planted-spillover simulations, or if its gene sets transfer worse to dissociated cohorts.

The strongest existing spatial-enrichment method is MONTAGE (MONTAGE). It multiplies a cell-type expression matrix by a neighborhood-proportion matrix to form spatial-community signatures, then tests functional gradients with local Moran's I. We adapt it by building the neighborhood-proportion matrix from spatial-transcriptomic cell types instead of CODEX proteomics. No reported result found on Lacuna for MONTAGE's sensitivity or specificity on a pathway-detection benchmark. Its reported evidence is 8 spatial communities in head and neck cancer and a survival association in more than 1,000 samples.

On dissociated data, the strongest enrichment baseline is DeepGSEA (DeepGSEA). It learns pathway-specific prototype embeddings and tests enrichment with Mann-Whitney U, and it was shown to be more sensitive than ssGSEA, GSVA, AUCell, Vision, SCPA and scGSEA on Splatter simulations. The file gives no exact sensitivity figure: no reported result found on Lacuna for DeepGSEA's numeric sensitivity. ResidPCA is the strongest state-discovery baseline. It reports more than 4 times the accuracy of PCA and more than 3 times that of NMF at recovering cross-cell-type states in simulation. We give DeepGSEA and ResidPCA the same dissociated cohorts and the same tuning budget as our transferred gene sets.

Three ablations isolate the key ingredients. The first removes the spillover term. The second replaces the deep niche encoder with the linear one. The third shuffles neighbors while preserving composition, which should drive every niche-attributable fraction to its null.

Why not just…?

Why not just run ORA on glial differentially expressed genes? It answers a different question: whether a pathway changes on average, not whether its change is set by the neighborhood. It also discards coordinated moderate shifts at arbitrary thresholds. In the file it produced no APOE or TREM2 signal in microglia on GSE243243 (re-analysis). It remains the floor in our experiments rather than a straw man.

Why not just regress AUCell scores on neighborhood composition? This is the real threat, and we treat it as one. If the linear AUCell model matches our attribution in the spatial simulations and in transfer, the contribution shrinks to the spillover correction and the calibrated null. We would then report it as such. We expect it to fall short in two places. First, leaked transcripts from adjacent astrocytes or neurons make neighborhood composition look predictive of microglial pathway scores. Our planted-spillover simulation measures exactly this false-positive rate. Second, whole-set scores cannot tell which member genes are context-driven, so they cannot produce the split gene sets that the transfer experiment tests.

Why not just use MONTAGE? MONTAGE finds where functional signatures vary across tissue. It does not separate a cell's intrinsic state from its neighbors' influence within a cell type, and it has only been demonstrated in cancer with matched proteomics. Its linear signature matrix is a fair competitor, and we run it as one. The decisive comparison is whether its spatial communities carry the same disease-stage signal once transferred.

Why not just apply ResidPCA and correlate its states with neighborhood afterward? ResidPCA removes cell-type baselines linearly. It can also erase states that are nearly collinear with one cell type, which is exactly where niche-driven glial activation would sit. Post hoc correlation with neighborhood also inherits the spillover confound. We include this two-step pipeline as a baseline on the spatial data.

Why not just pool scMamba embeddings over spatial neighbors? This would give a context-aware cell representation. But it attributes nothing to curated pathways, and scMamba was trained on dissociated nuclei, so its embeddings would absorb spillover without flagging it. We test it as an alternative niche encoder inside our estimator. If it helps, that is a component improvement, not a competing answer.

Proposed Experiments

The first experiment is a controlled simulation. We extend Splatter, the simulator DeepGSEA used, with spatial layouts. We plant three kinds of genes: niche-induced programs in microglia driven by astrocyte density or plaque distance, cell-intrinsic programs, and contact-weighted spillover of neighbor markers. The metrics are sensitivity and false discovery rate for niche-attributable pathway calls, and uniformity of p-values under a no-niche null. The decisive check is whether the method keeps false discoveries at the nominal rate under strong spillover, and whether the AUCell-regression and MONTAGE baselines do not.

The second experiment tests spatial replication. In the Mouse Brain Aging series used by NicheFlow, and in any available human Alzheimer's spatial cohort, we call niche-attributable pathways on some slides and test whether they replicate on held-out slides and donors. We report the replication rate for each method. Tuning $R$ and $K$ on the training slides only guards against neighborhood-definition shopping. A negative control asks whether pathways for exclusive neuronal markers are called niche-attributable in microglia after the spillover correction. They should not be.

The third experiment is the payoff. We score the split gene sets in the Morabito cohort, the scMamba compendium cohorts, and GSE243243. We test association with pathology stage within microglia and astrocytes, using donor-level mixed models, and compare against full curated sets, DeepGSEA, and ResidPCA states. We also check whether donors with high niche-driven scores overlap the ResidPCA heritability-enriched axis. The proposal succeeds if the niche-driven halves show stronger, cross-cohort-replicating stage associations than the full sets and the intrinsic halves. It fails if the intrinsic halves carry the signal, a result that would itself argue against strong non-cell-autonomy for those pathways.