Spatial pathology routinely asks a statistical question it rarely answers rigorously: is the mix of cell types around one kind of structure (a tumor margin, a blood vessel, a K17-positive tumor nest) different from the mix around another? Each local neighborhood is a vector of cell-type proportions, which is compositional data: non-negative parts that sum to one. These vectors are mostly zeros, because rare cells such as regulatory T cells or mature dendritic cells are absent from most neighborhoods. A single slide also yields hundreds of thousands to millions of neighborhoods that overlap and are spatially correlated. Current practice either ignores the simplex geometry, using raw counts or Euclidean scores, or takes logarithms of proportions after adding an arbitrary small "pseudocount" to the zeros. It then computes p-values as if neighborhoods were independent. The first choice distorts distances. The second makes nearly any difference look significant.
We propose a two-sample test for whole-slide neighborhood compositions that does three things. It handles zeros without pseudocounts by projecting proportions onto a sphere. It reduces to sums of explicit feature vectors, so it runs in one pass and merges across machines with a single exchange of statistics. It corrects its variance estimate for spatial dependence using tile-level blocks. Because the test reduces to sufficient statistics, the point at which distributing the work beats a single GPU can be predicted analytically. We expect that point to lie beyond the size of most cohorts, which is itself a useful negative finding for practitioners tempted to reach for a cluster.
Core Claim
The method builds on the studentized spherical-harmonics test of A Studentized Spherical Harmonics–Based Nonparametric Two-Sample Test for Compositional and Directional Data. That test splits functions on the sphere into finite-dimensional pieces using spherical harmonics, the sphere's analogue of Fourier modes. From them it builds a kernel energy distance, a measure of how far apart two distributions are. It then standardizes the statistic so that under "no difference" it is approximately standard normal. The result needs no kernel width to tune and no permutations. The paper reports 100% power where centered-log-ratio methods reach 5% in its Dirichlet-multinomial simulations. It was validated only on small independent samples (60 throat microbiomes), and the research file flags that its asymptotics need sample sizes that can exceed $2^d$ in dimension $d$.
Spatial tissue reverses that problem. Sample sizes are enormous, so the $2^d$ requirement is met easily for 27 IMC phenotypes. Independence fails badly instead. Our claim is that two changes make the test usable at whole-slide scale. The first is to replace the independent-sample variance with a variance computed from spatial tiles. The second is to compute the statistic through explicit harmonic features up to a fixed degree. With both changes, the test is calibrated, meaning false-positive rates match the nominal level, on overlapping neighborhoods where the original test is not. It also keeps its power advantage over pseudocount log-ratio tests on rare-phenotype differences.
Zeros are handled by the radial map $\psi(x)=x/\|x\|_2$ from Kernel Methods for Radial Transformed Compositional Data with Many Zeros. That paper shows the map keeps the simplex boundary at finite distance and preserves ratios between parts, and that it separates zero-heavy clusters which centered log-ratios smear together. Here $\psi$ supplies the geometry rather than the method.
Research Program
For each anchor cell, the neighborhood composition $x$ is the vector of cell-type proportions within a fixed radius. The proportions come from segmentation and classification already produced by the source datasets. Write $\phi(\cdot)$ for the vector of orthonormal spherical harmonics up to degree $p$, evaluated at $\psi(x)$.
For two neighborhood populations $A$ and $B$, such as neighborhoods within 100 µm of K17+ versus K17− nest boundaries, let $\bar\phi_A$ and $\bar\phi_B$ be the mean feature vectors. The statistic is a bias-corrected squared distance between these means, divided by an estimated standard error:
$$ Z=\frac{\|\bar\phi_A-\bar\phi_B\|^2-\hat b}{\hat\sigma_{\text{block}}}. $$
Here $\hat b$ removes the self-pairs that a U-statistic excludes. $\hat\sigma_{\text{block}}$ is estimated by grouping neighborhoods into square tiles larger than the spatial correlation length and treating tile-level sums, rather than individual neighborhoods, as the independent units. The tile side is chosen from the decay of the empirical autocorrelation of $\phi$. We will report how sensitive results are to that choice.
We will derive the finite-feature form as our own contribution. It follows because each finite-degree harmonic space has an explicit orthonormal basis, so the kernel restricted to degree $\le p$ is an inner product of $\phi$ vectors. Its consequence is that every quantity in $Z$ is a sum over neighborhoods or tiles of $\phi$, $\|\phi\|^2$, and tile-sum outer products. This is the same "compute summary statistics locally, then combine" pattern that Large-scale distributed non-negative sparse coding and sparse dictionary learning used to turn sparse coding into single-pass MapReduce jobs. We reuse the pattern, not the model.
For cohort-level questions, such as anti-PD-1 responders versus non-responders, the patient is the unit of inference. Each patient's $\bar\phi$ becomes a point, and the same standardized statistic is applied across patients. This connects to the per-region signatures of Graph of graphs analysis for multiplexed data with application to imaging mass cytometry. That work compares probability-vector signatures with Euclidean heat kernels; we compare them with a geometry that respects zeros.
The systems component is a cost model rather than a new distributed engine. Let $N$ be the number of neighborhoods, $F$ the feature dimension, $c$ the per-feature evaluation cost, $K$ the number of workers, $T_0$ the fixed start-up cost, and $T_r(F)$ the time for one all-reduce of an $F$-length vector. Distribution pays off when
$$ cNF\left(1-\tfrac{1}{K}\right) > T_0 + T_r(F), \quad\text{that is,}\quad N^\ast=\frac{T_0+T_r(F)}{cF(1-1/K)}. $$
This follows from the serial-versus-distributed time decomposition summarized in the file. It differs from the iterative, synchronization-heavy settings of MegBA: A GPU-Based Distributed Library for Large-Scale Bundle Adjustment and the fine-grained pipelining of HARMONY: A Scalable Distributed Vector Database for High-Throughput Approximate Nearest Neighbor Search. Both pay communication on every step, and the HARMONY authors note this overhead swamps gains on small data. Our test communicates exactly once. We will measure $c$, $T_0$ and $T_r$ on single GPU, multi-GPU and Spark configurations, then check whether measured crossover sizes match $N^\ast$.
The feature dimension grows roughly like $d^p/p!$, which gives about $2\times10^4$ for $d=27$, $p=4$. This is the main scaling risk. We will study power as a function of degree $p$, and an amalgamated variant that merges unselected phenotypes into one residual part in the style of Kernel Sufficient Dimension Reduction and Variable Selection for Compositional Data via Amalgamation.
Baselines
The naive reference is what spatial papers do today. We add a pseudocount to neighborhood proportions, apply the centered log-ratio, and run a Hotelling or Euclidean energy test that treats neighborhoods as independent. We run it at several pseudocount values, because its sensitivity to the pseudocount is part of the evidence. Beating it means comparable or better power together with calibrated false-positive rates, and stable conclusions across pseudocounts where the baseline's conclusions flip.
The domain baseline is the influence-zone analysis of A Novel Framework for Characterization of Tumor-Immune Spatial Relationships in Tumor Microenvironment. It counts each immune subset per unit area within 100 µm of tumor nests and compares the result against random spatial simulations using mean absolute deviation, analyzing one subset at a time. It reports deviations of 1.36 for CD8 T cells and 0.99 for CD163 macrophages on its 23 PDAC slides. We rerun it on the same slides and cell calls. Beating it means detecting the joint compositional shift between K17+ and K17− margins, including shifts in subsets it reports as weaker (CD4, CD16), with a p-value whose calibration we can verify on label-shuffled slides.
The strongest existing method is the original studentized spherical-harmonics test applied directly to neighborhoods with its independent-sample variance. It reaches 100% power versus 5% for centered log-ratio in its own simulations. No reported result was found on Lacuna for its false-positive rate under spatial dependence. Running it on identical features and degree $p$ makes it the key ablation, since the only difference from our method is block studentization. We expect it to be badly anti-conservative on overlapping neighborhoods. Our method beats it if it holds nominal false-positive rates while losing only modest power.
A second ablation replaces the radial map with the square-root map. This isolates the zero-handling ingredient from the testing machinery.
For cohort prediction, used only as a sanity check that the embeddings carry signal, the reference points are the following. The graph-of-graphs method reports ROC AUC 0.945 on NSCLC cohort 1 and about 0.83 for a direct diffusion-map baseline. Cox-nnet on proximity scores in Deep learning and transfer learning identify breast cancer survival subtypes from single-cell imaging data reports a C-index of 0.80 versus 0.63 for clinical subtyping. We do not claim to beat these predictors; testing is the goal.
All methods get the same cell calls, neighborhood radius and compute budget. Any tuned parameter (pseudocount, kernel width, tile size) gets the same grid size.
Why not just…?
Why not just add a pseudocount and use log-ratios? The radial-kPCA paper shows that pseudocount log-ratios smear simulated zero-heavy clusters. On the Hayden microbiome data they need 128 principal components to explain 90% of variance, against 35 for the radial map. The spherical-harmonics paper reports 5% power for log-ratio tests where its own test reaches 100%. Our pseudocount sweep tests directly whether these failures carry over to immune neighborhoods. If log-ratio tests on abundant phenotypes perform as well as ours, we will report that the gain is confined to rare-phenotype contrasts.
Why not just use a Gaussian-kernel maximum mean discrepancy with random Fourier features and a spatial block permutation? This is the real threat. It is also linear-time, also merges across machines, and a permutation over tiles also handles dependence. Our method differs in two ways. It needs no kernel width, and it gives an analytic p-value instead of hundreds of permutation passes over millions of neighborhoods. Whether either matters is empirical. We will run this competitor under the same tiles and report power, calibration and wall-clock time. If it matches us on all three, the contribution narrows to the geometry of the radial map, and we will say so.
Why not just subsample neighborhoods until they are roughly independent? Thinning to well-separated anchors restores independence but discards most of the data. The file's evidence on dendritic-cell huddling (Chang et al.) and tertiary lymphoid structures (Larsson et al.) suggests the signal often lives in small, fragile structures. We include thinning as a baseline and compare its power curve against block studentization.
Why not just run it on Spark? Road to Freedom in Big Data Analytics found a plain Java program nearly 10× faster than Spark for SVM training on small benchmarks. Our statistic is a single sum. We therefore expect a single GPU to win up to $N^\ast$, and we report the measured crossover rather than assuming distribution helps.
Proposed Experiments
The first decisive check is calibration. On real slides, we shuffle the region labels at the tile level, which preserves spatial structure while destroying any true difference. We also simulate spatially correlated multinomial neighborhoods with planted rare-phenotype zeros. Across thousands of null replicates we measure the false-positive rate at 0.05 for our method, the original independent-sample test, and the pseudocount log-ratio test. The proposal fails if block studentization does not bring the rate near nominal.
The second check is power on planted differences. We shift the abundance of one rare phenotype, at varying sparsity levels, between two regions of real IMC breast cancer slides, and compare power at matched calibration.
The third is application. We contrast K17+ and K17− margins on the PDAC slides, compare tumor-adjacent stroma against vessel-adjacent stroma in the IMC breast cohort, and test responders against non-responders across patients in the two NSCLC cohorts. In the NSCLC cohorts the patient is the unit, so sample sizes are only 55 and 29, and we will state power limits honestly.
The fourth is systems validation. We sweep $N$ from $10^4$ to $10^8$ neighborhoods by pooling slides and synthetic tiling. We time single GPU, 8-GPU all-reduce and Spark, then compare the measured crossover against the predicted $N^\ast$ and against the cost of the random-Fourier-feature permutation competitor. The headline result is either "one GPU suffices for every cohort in the file" or a measured crossover size where it no longer does. Both outcomes are informative.
References
- A Studentized Spherical Harmonics–Based Nonparametric Two-Sample Test for Compositional and Directional Data
- Kernel Methods for Radial Transformed Compositional Data with Many Zeros
- Large-scale distributed non-negative sparse coding and sparse dictionary learning
- Graph of graphs analysis for multiplexed data with application to imaging mass cytometry
- MegBA: A GPU-Based Distributed Library for Large-Scale Bundle Adjustment
- HARMONY: A Scalable Distributed Vector Database for High-Throughput Approximate Nearest Neighbor Search
- Kernel Sufficient Dimension Reduction and Variable Selection for Compositional Data via Amalgamation
- A Novel Framework for Characterization of Tumor-Immune Spatial Relationships in Tumor Microenvironment
- Deep learning and transfer learning identify breast cancer survival subtypes from single-cell imaging data
- Spatial organization of dendritic cells within tumor draining lymph nodes impacts clinical outcome in breast cancer patients
- Spatial deconvolution of HER2-positive breast cancer delineates tumor-associated cell type interactions
- Road to Freedom in Big Data Analytics