Many gene-expression studies in cancer compare tumors that a pathologist placed in one category against tumors placed in another: Gleason grade 3 versus 4, receptor-positive versus negative, dysplasia versus regenerative change. The standard statistical test for which genes differ between the groups is called differential expression (DE) analysis. It treats those category labels as exact truth. Pathologists, however, often disagree on exactly these calls. When a six-pathologist panel splits on a prostate core, the "grade 4" group used in a DE analysis is partly made of grade-3-like tissue. That contamination dilutes real signals and can produce spurious ones.
This proposal treats each pathologist's vote as a noisy measurement of an unobserved true class and folds that uncertainty directly into the count model used for DE. It then asks two questions. The first is whether this recovers more correct genes than majority vote. The second is whether the corrected gene lists agree better with an independent, label-free referee. That referee is the set of genes that the defining transcription factor (a protein that binds DNA to switch genes on) actually binds in open chromatin in the relevant cell type. Chromatin is the DNA-protein packaging of the genome, and "open" means accessible enough for such binding. A secondary analysis uses co-expression modules to ask what ambiguous cases look like transcriptionally. Do they sit on an intermediate continuum, are they mixtures of two clear phenotypes, or are they transcriptionally unambiguous, so that the disagreement is purely perceptual?
Core Claim
The file identifies a concrete blind spot. DE workflows assume group labels are ground truth, and this assumption breaks when labels come from human observers. Work on observer variability has modeled disagreement only in image space. The Latent Doctor Model uses a conditional variational autoencoder over annotator styles to predict the full distribution of a pathology panel's grades. It reports a KL divergence of 0.461 on Gleason2019 and a Spearman correlation of 0.728 between its predictive entropy and the entropy of a 20-pathologist ISUP panel. Meanwhile, co-expression studies such as Lian et al. correlate gene modules with single-consensus clinical stages and assume those stages are noise-free. No one in the file connects the two.
Our claim is that a DE estimator marginalizing over a latent true class will produce better-calibrated false discovery rates and more accurate gene rankings than majority-vote DE. The latent class is informed by per-annotator votes and, optionally, by an image model, but never by the expression data being tested. A second claim is methodological. Accessibility-gated transcription-factor target sets provide a usable external referee for DE on real data where no clean labels exist, whereas motif-only target sets are too contaminated to serve that role. The asymmetry is expected because genome-wide motif scanning yields large numbers of inactive sites (Asap), and motifs in closed chromatin often have no regulatory effect (Chan et al.).
Research Program
The estimator works as follows. For sample $i$, let $z_i$ be the unobserved true class. Each annotator $j$ casts a vote $v_{ij}$ drawn from a per-annotator confusion matrix $\pi_j$, in the style of standard latent-annotator models. Read counts $y_{ig}$ for gene $g$ follow a negative binomial distribution. This is the overdispersed count model used by DESeq-style tools (Yang et al.). Its mean is $\mu_{igk} = s_i \exp(\beta_{g0} + \beta_{gk})$ under class $k$, where $s_i$ is a library-size factor and $\phi_g$ is a dispersion. The per-gene likelihood is
$$ L_g = \prod_i \sum_k w_{ik}\, \mathrm{NB}(y_{ig}\mid \mu_{igk}, \phi_g), $$
where $w_{ik} = P(z_i = k \mid v_{i\cdot}, \text{image}_i)$ is computed only from votes and an image prior. The DE test is a likelihood-ratio test on $\beta_{gk}$, followed by Benjamini-Hochberg correction (Li et al.).
Excluding expression from $w_{ik}$ is a deliberate guard against double-dipping. If expression shapes the class posterior, genes become "differential" partly because they moved the labels. We will build that variant anyway and show its false discovery rate inflation, so the guard is justified empirically rather than asserted.
The referee is built from context-dependent binding. For a label defined by a transcription factor, its targets are genes with that factor's motif inside accessible chromatin of a matched cell type. Accessibility comes from ENCODE/ROADMAP DNase or ATAC data. The construction follows SPIDER, which seeds a message-passing regulatory network with motifs restricted to open chromatin and recovers binding at AUC 0.82–0.85. We contrast this with a motif-only target set.
The cleanest test case is estrogen receptor status in breast cancer. There the class label is literally the activity of a transcription factor (ER, encoded by ESR1). The label is known to be unstable in the ER-low range, and Shamai et al. found 31 IHC misdiagnoses using an H&E model with ER AUC 0.95. The file's ENCODE cell-line list (GM12878, K562, HepG2, A549, H1, HeLa) does not include a breast line. Securing ER-relevant accessibility data is therefore a prerequisite. If none is available, the referee falls back to SPIDER-style networks on the listed lines, applied only to labels whose driving factor is active in them.
The module analysis applies standard co-expression network construction to each cohort: soft-thresholded correlation, topological overlap, and module eigengenes, which are first principal components summarizing a module. For panel-ambiguous samples, we ask where the eigengenes of grade-associated modules fall. They may be intermediate between class centroids, which would indicate a continuum. They may be bimodal across modules, which would indicate a mixed lesion. Or they may be cleanly aligned with one class, which would indicate perceptual disagreement over transcriptionally clear tissue.
Bulk data cannot separate "intermediate" from "mixed" within a single sample, because cell-composition shifts mimic both. Where HEST-1k includes prostate spatial transcriptomics (HEST-1k), we will check whether ambiguous-looking regions show within-region heterogeneity of module scores.
Baselines
The do-nothing reference is negative binomial DE on majority-vote or report labels. This is what almost every study in the file does, for example the TCGA risk-model papers (PLK1, OSCC).
The strongest simple competitor is agreement-weighted DE. Each sample keeps its majority label but receives a precision weight equal to the panel's vote fraction for that label. It uses the same NB model, the same normalization, and the same genes. It has no tuning parameters to favor either side.
A second strong competitor is consensus-only DE, which discards every sample below a fixed agreement threshold. We sweep the threshold over the same grid used for any hyperparameter of our method.
The ablation that isolates the key ingredient is our mixture model with $w_{ik}$ taken from votes alone, compared against votes plus the image prior. This tells us whether the image model adds information beyond the panel.
For the panel simulator, the strongest existing model is the Latent Doctor Model. Its reported figures are KL 0.461 on Gleason2019 and Spearman 0.728 on ISUP panel entropy. Our reimplementation must reach comparable figures before we use it to impute vote distributions.
For DE accuracy under observer noise there is no reported result on Lacuna, so all DE baselines are run by us. Beating them means two things. First, observed false discovery proportion must stay within nominal FDR where the majority-vote baseline exceeds it. Second, recall of the top clean-label genes must be higher at matched FDR, with paired bootstrap confidence intervals excluding zero difference against agreement-weighted DE.
Why not just…?
Why not just drop the ambiguous samples? This is the most serious threat. If disagreement is concentrated in a small fraction of cases, consensus-only DE loses little power and is trivially simple. Gleason2019's per-core six-pathologist votes and ISUP's 20-pathologist votes let us measure how large that fraction actually is. The semi-synthetic experiment reports power as a function of the discarded fraction. If consensus-only matches our method at realistic disagreement rates, we will report that as the main finding: a practical recommendation, not a new estimator.
Why not just weight samples by panel agreement? Weighting down-weights doubtful samples but still assigns them to the wrong side when the minority vote is correct. That is exactly the ER-low regime, where Shamai et al. found IHC calls overturned on re-staining. The mixture model lets such samples contribute to the correct class. Whether this matters is an empirical question. Agreement-weighted DE is a headline baseline, and if it ties, the contribution shrinks to the referee and the module analysis.
Why not just replace labels with a continuous molecular score, such as ESR1 expression or a module eigengene? This avoids observer noise entirely, but it changes the question. Genes then become "differential" with respect to a quantity defined from the same expression matrix. That analysis is circular for discovery, and it no longer studies the clinically used phenotype. We include it as a reference in the module analysis rather than as a DE competitor.
Why not just use motif-scanned targets as the referee? Motif-only sets include many inactive sites (Asap; Chan et al.). Adding accessibility improved binding auPRC by up to 15 points over sequence-only models in EpiBinder. We test the referee choice directly. In semi-synthetic data, where the true DE ranking is known, we compare how well each target set's enrichment score ranks the DE methods in their true order.
Proposed Experiments
The first experiment is semi-synthetic and decisive for the estimator. We take TCGA cohorts with paired bulk RNA and slides and fix their existing labels as truth. We then corrupt them with annotator confusion matrices estimated empirically from Gleason2019 and ISUP panels, at panel sizes from three to twenty. Every DE method is scored against the clean-label result on false discovery calibration and top-gene recall. The double-dipping variant is included to show its inflation.
The second experiment is real-data prostate. We predict panel vote distributions on TCGA slides with the reimplemented Latent Doctor Model. We validate them against a small newly collected multi-pathologist panel on a subset of slides, and we state openly that this collection is a cost of the proposal. We then compare DE methods on accessibility-gated androgen-receptor-related targets, conditional on obtaining suitable accessibility data.
The third experiment is breast ER-status DE. Here the label-free referee is most natural, and an H&E ER model in the style of Shamai et al. serves as the image prior.
The final check is the module analysis of ambiguous cases. The method is judged useful only if two things hold together: the semi-synthetic experiment shows calibrated FDR where majority vote fails, and real-data enrichment on context-gated targets ranks the methods in the same order the semi-synthetic truth does.
References
- The Latent Doctor Model for Modeling Inter-Observer Variability
- The co-expression networks of differentially expressed RBPs with TFs and LncRNAs related to clinical TNM stages of cancers
- Asap: A Framework for Over-Representation Statistics for Transcription Factor Binding Sites
- Impact of cancer mutational signatures on transcription factor motifs in the human genome
- Transcriptomic Analysis of the Candidate Genes Related to Aroma Formation in Osmanthus fragrans
- Identification of differentially expressed genes and signaling pathways in chronic obstructive pulmonary disease via bioinformatic analysis
- Constructing gene regulatory networks using epigenetic data
- Clinical utility of receptor status prediction in breast cancer and misdiagnosis identification using deep learning on hematoxylin and eosin-stained slides
- HEST-1k: A Dataset For Spatial Transcriptomics and Histology Image Analysis
- PLK1 overexpression as a dual-role biomarker and therapeutic vulnerability in pulmonary adenocarcinoma
- A Risk Score Model Based on Drug-Sensitivity-Related Genes Has the Potential to Predict Oral Squamous Cell Carcinoma Prognosis.
- EpiBinder: a multimodal deep learning model at base-resolution to analyze in vivo Transcription Factor-DNA Binding