Cell-to-Cell Crosstalk Cancellation for Multiplex Immunohistochemistry Scoring

Pathologists score protein biomarkers by staining tissue and counting how many cells light up and how strongly. One common score, the H-score, weights each cell by its staining intensity and sums the result. When six or more colored stains share one brightfield slide, each cell's measured color stops being its own. It blends the cell's own dyes with dye from the membranes and cytoplasm of the cells pressed against it, and the overlapping color spectra then blur the result further. Current deep-learning approaches treat this as a per-pixel color-separation problem. Wireless engineers have solved a problem of the same shape for decades: many transmitters share a medium, each receiver hears a superposition, and the fix is to model who interferes with whom as a graph and cancel the interference in a learned, structured way. This note proposes to carry that machinery into tissue. Each segmented cell becomes a node, physical adjacency defines interference edges, and per-cell stain amounts are recovered by an unrolled successive-cancellation network rather than by a generic image model.

The payoff is a per-cell stain estimate that remains reliable in crowded tissue, such as immune cells pressed against tumor nests. That reliability matters because both bulk scores like the H-score and spatial biomarkers like immune-to-tumor distances are built on these estimates. A secondary question is clinical. If crosstalk errors depend on tissue density, they could create spurious associations between biomarkers and patient age or sex, and the field relies on exactly such associations. We test whether cancelling crosstalk changes those associations.

Core Claim

Optical crosstalk in brightfield multiplex IHC is structured. It flows along cell contacts, and it is roughly additive in optical density, because under the Beer–Lambert law absorbance adds across overlapping dye layers. Under this structure, a few layers of an unrolled interference-cancellation estimator should recover per-cell marker identity and intensity better than per-pixel unmixing. They should also beat a generic graph network of matched size, and the gain should concentrate in dense neighborhoods. Concretely, we model the optical density measured over cell $i$ as

$$y_i = A c_i + \sum_{j \in \mathcal{N}(i)} g_{ij} A c_j + n_i,$$

where $c_i$ is the vector of six stain concentrations belonging to cell $i$, $A$ is the stain color matrix that maps concentrations to RGB optical density, $\mathcal{N}(i)$ is the set of cells touching or near cell $i$, $g_{ij} \ge 0$ is a learned spillover gain from cell $j$ into cell $i$ that depends on contact length and distance, and $n_i$ is residual noise. The problem has the form of a multi-user interference channel: $A c_i$ is the desired signal, the sum is interference, and estimating $\{c_i\}$ is joint detection.

Relation to Prior Work

The method comes from wireless resource management. Unfolding WMMSE Using Graph Neural Networks turns the iterations of a classical power-allocation algorithm into a few neural layers whose weights a graph network produces over the interference graph. It reports a sum-rate of 83.21 in 4 layers against 82.94 for 100 classical iterations, at 2 ms versus 16 ms. We reuse the unrolling idea but change the target. The quantity being unrolled is not a power allocation but successive interference cancellation: decode a signal, subtract its contribution from its neighbors, and repeat.

Two theoretical results shape how we design the estimator. Shen et al. show that graph networks are equivalent to a class of distributed optimizers, which justifies expecting a shallow unrolled network to approximate the iterative estimator. Comaniciu's result that, under channel-estimation error, cancellation order should follow estimation reliability rather than received power (author page) gives us a specific design choice. At each layer we cancel the most confidently estimated cells first, not the darkest. In tissue the two orderings differ: dark pigment or dense stroma is intense but unreliable.

Geng et al. prove that treating interference as noise is optimal only when cross-links are weak. That result motivates the whole question, since it predicts our method helps only where cells are densely packed and stays neutral in sparse tissue. We test that prediction directly.

On the pathology side, Fassler et al. are the closest work and our main competitor. They separate six chromogens (CD3, CD4, CD8, CD20, CD16, K17) in pancreatic cancer using a color autoencoder and a U-Net, trained from sparse seed labels expanded by superpixels. They report per-cell-type F1 of 0.63–0.83 and then compute nearest-neighbor distances, for example a median of about 2.9 μm from CD16+ myeloid cells to tumor cells. Their models decide per pixel within a convolutional receptive field. They have no explicit notion of which neighboring cell is contaminating which. The very small myeloid-to-tumor distance they report is also the regime where we expect crosstalk to be worst.

The HCC cell-graph study builds cell graphs, which it connects by minimum spanning tree, and runs heterogeneous message passing to predict recurrence (C-index 0.823). It uses the graph to summarize biology, not to correct measurement, and its tree edges would drop many of the contacts that carry crosstalk. Scalar scores such as the H-score, used for Trop-2 (Dopfer et al.) and PARP1 (Kupczyk et al.), inherit whatever per-cell errors sit beneath them.

The demographic angle is motivated by the pharmacovigilance and EHR literature. Xia et al. find older and male patients at higher risk of cardiac checkpoint-inhibitor toxicity. Wang et al. find younger and female patients at higher overall immune-adverse-event risk. Both remark that the cellular mechanism is unobserved. Tissue-level immune measurements are an obvious candidate mechanism, so their measurement error matters.

Research Program

The work has three parts.

The first part is building the cell interference graph. We segment nuclei and approximate membranes on the multiplex slide. Edges connect cells within a contact radius, and each edge carries contact-length and center-distance features. Following the wireless setup, every cell is one node, and edge features play the role of channel gains.

The second part is the unrolled cancellation estimator, which runs in three stages. A shared per-cell encoder produces an initial concentration estimate and a confidence for each cell. Each of $K$ layers, with $K$ between 3 and 6, then re-estimates spillover gains from edge features and current estimates, and subtracts the predicted contribution of neighbors with weights that grow with neighbor confidence. Finally, the estimate is refined through a nonnegativity-constrained update derived from the model above, following the reliability-ordered cancellation rule. Weights are shared across nodes, so the network is unchanged when cells are relabeled. That property gives it the same zero-shot transfer to larger graphs that made graph networks attractive in wireless.

Training combines Fassler-style seed labels with a self-supervised term. That term requires the reconstructed slide, $A c_i$ plus predicted spillover, to match the observed optical density. Because the forward model is physical, this term supplies supervision where no labels exist.

The third part is downstream scoring. From the corrected per-cell concentrations we compute per-marker H-scores and nearest-neighbor distance statistics. We also compute a contamination map, the estimated interference-to-signal ratio for each cell. This is the tissue analogue of a signal-to-interference ratio, and it tells a pathologist where a score should not be trusted.

Baselines

The do-nothing reference is classical color deconvolution with a fixed stain matrix, followed by per-cell averaging. This is what most open tools do, and it is equivalent to treating neighbors' dye as noise. Fassler et al. report that standard deconvolution breaks down with six overlapping brightfield stains. We include it anyway to calibrate how much any method gains.

The strongest existing method is the ColorAE:U-Net ensemble of Fassler et al., which reports F1 of 0.63–0.83 across six cell types on its PDAC 6-plex data. It is the only cited system built for exactly this task. We retrain it on identical training slides, seed labels, and superpixel expansion, with the same GPU budget and the same number of hyperparameter trials as our model. Beating it means a higher macro F1 on the same held-out cells, with the gain significant under a slide-level bootstrap. We also expect the gain to sit mainly in the dense-contact stratum. A uniform gain would suggest we had merely built a better image model.

The key ablation is a generic graph network of matched parameter count, built on the same cell graph and initial features but with free-form message passing instead of explicit subtraction and confidence ordering. This ablation separates "a graph helps" from "structured cancellation helps." Two smaller ablations swap reliability-ordered cancellation for intensity-ordered cancellation, which tests the Comaniciu-derived choice, and replace contact edges with the minimum-spanning-tree edges used in the HCC cell-graph study.

We also run a classical non-learned iterative solver for the same forward model: nonnegative least squares with alternating spillover estimation, run to convergence. It is the tissue analogue of running WMMSE for 100 iterations. It tells us whether learning adds anything beyond the physics model. We have no reported number on Lacuna for this solver on multiplex IHC.

Why not just…?

Why not just make the U-Net bigger or give it a wider receptive field? A wider convolutional field can in principle see neighbors. But it has no reason to attribute dye to a particular neighboring cell, and it must learn the additive mixing structure from sparse seed labels that Fassler et al. had to expand with superpixels. This alternative is a real threat, so we test it directly. We train a U-Net with dilated layers whose receptive field matches our graph's two-hop neighborhood. If it closes the gap in the dense-contact stratum, the graph formulation is unnecessary, and we will say so.

Why not just run a generic cell-graph network on top of the U-Net outputs? This is the most plausible competitor, because the HCC work shows such networks work well on cell graphs. The matched-size generic-network ablation addresses it. Our claim survives only if explicit subtraction plus confidence ordering wins at equal capacity, or matches accuracy while needing fewer labels. We therefore also report learning curves at 10%, 30%, and 100% of seed labels. The physics-shaped model should be most valuable when labels are scarce.

Why not just solve the forward model classically, without learning? If the Beer–Lambert spillover model were exact, a classical solver would suffice. In practice the stain matrix drifts across slides and centers. The Ki67 domain-shift work of the domain-specific pruning study documents this kind of protocol and scanner variation. Spillover also depends on section thickness. The learned gains are meant to absorb this. The classical solver baseline measures how much of the gain comes from the model and how much from the learning.

Proposed Experiments

The first experiment is a semi-synthetic benchmark with exact ground truth. We place real segmented cell layouts from the PDAC and Ki67 data at controlled densities, assign known six-marker concentrations, render optical density through the Beer–Lambert model with spatial spillover and scanner blur, and vary the spillover strength. This reproduces the weak-to-strong interference sweep from the wireless literature. The prediction from Geng et al. is decisive here. All methods should tie when spillover is weak, and our margin should grow with density and spillover strength. If no such crossover appears, the core claim fails.

The second experiment uses real multiplex data. It requires obtaining the Fassler et al. 6-plex PDAC slides and seed labels. We report per-cell-type F1 against their 0.63–0.83, stratified by number of contacting cells. We also check whether the reported ~2.9 μm myeloid-to-tumor distance and the gap between myeloid and lymphoid distances persist after crosstalk is removed. If the gap shrinks substantially, part of that published spatial finding may be a measurement artifact, which would be a useful result whichever way it falls.

The third experiment tests transfer. We train on small tissue regions and evaluate on whole slides, and we train on a subset of scanning domains in the 16-domain Ki67 data and test on held-out ones, with the gains of domain-specific pruning over ERM (+0.35% detection F1 and +1.56% classification F1 on unseen domains) as the reference scale for meaningful generalization gains. The Ki67 data are singleplex, so here the cancellation acts on nucleus-to-nucleus bleed rather than cross-stain mixing. We state that limitation openly.

The fourth experiment is the demographic consistency check, applied wherever slides carry age and sex. The HCC whole-section cohorts include clinical covariates. For multiplex data with demographics, availability remains to be confirmed. We fit the same regression of immune-proximity and H-score features on age and sex, once with uncorrected and once with corrected per-cell estimates, and test whether the coefficients move beyond bootstrap noise. We also check whether contamination maps differ systematically by demographic group. The file contains no reported numbers for demographic associations on these imaging cohorts, so this experiment is exploratory and pre-registered as such. A null result would mean existing tissue-level demographic findings are robust to crosstalk, and that is also worth knowing.