Replicable Pathway Factors, Not Shared-Gene Artifacts

Gene-set enrichment analysis is the standard way to read a list of disrupted genes. It asks which known pathways (curated lists of genes that work together) are over-represented, and it routinely returns hundreds of significant pathways that are mostly restatements of a few processes. Every existing cleanup tool decides which pathways are "the same" from a fixed similarity score: how many genes two pathways share, how close they sit in the Gene Ontology hierarchy, or how similar their text descriptions are. None asks the question that matters to a biologist: across patients, do these pathways actually rise and fall together, and does that co-movement hold up in an independent cohort? This proposal treats pathway redundancy as a measurement problem. Each pathway's per-patient activity score is an "item," as in a questionnaire. Groups of pathways that measure one underlying process are "factors." The psychometric discipline of discovering factors in one sample and confirming them in another becomes the standard for which summaries of enrichment output can be trusted.

The twist is that pathway scores have a known, computable source of fake correlation. Two pathways that share genes will correlate across patients even when nothing biological links them, much like two survey items that reuse the same wording. We propose to subtract this shared-gene correlation explicitly inside the factor model, to use the Gene Ontology hierarchy to fix which way the factors are rotated, and then to test on held-out cohorts which notion of relatedness actually replicates: gene overlap, ontology hierarchy, or co-variation found in the data.

Core Claim

Two pathways are functionally redundant when they share latent variance across patients beyond what their shared genes force on them, and when that shared variance reproduces in independent cohorts. Our method makes this definition operational. It is a factor model of pathway activity scores, $\mathbf{S}$, an $N \times P$ matrix of $N$ patients by $P$ enriched pathways. The model is

$$\Sigma = \Lambda \Phi \Lambda^{T} + \Psi + \Omega,$$

where $\Lambda$ holds pathway-to-factor loadings, $\Phi$ is the correlation among factors, $\Psi$ is diagonal pathway-specific noise, and $\Omega$ is a fixed off-diagonal matrix of correlation caused by shared genes. If a pathway score is the average of its member genes' standardized expression, then shared genes contribute covariance $\sum_{g \in A\cap B} \sigma_g^2 / (|A||B|)$ between pathways $A$ and $B$, with $\sigma_g^2$ the variance of gene $g$. Under independent unit-variance genes this reduces to $|A\cap B|/\sqrt{|A||B|}$, which follows directly from the variance of a sample mean. In practice we estimate $\Omega$ from the discovery cohort's gene covariance, including cross terms between shared and unshared genes, and hold it fixed thereafter. Psychometrics calls this a correlated-uniqueness term. Here, unusually, its values can be computed rather than estimated.

The hierarchy enters at the step where factor analysis is weakest. Exploratory factor analysis (EFA) has no unique solution, and rotation heuristics such as Varimax can give different biological stories. We instead rotate toward a target loading pattern built from information-content similarity on the Gene Ontology. Information-content similarity rates two terms as close when their most specific shared ancestor is itself specific. It works natively on the ontology's multi-parent graph and needs no tree assumption. The rotation is a soft prior, not a constraint. Where the data disagree with the ontology, the disagreement is visible in the loadings and becomes a finding.

Positioning Against Prior Work

The redundancy literature motivates this work and supplies the competing similarity notions. Set-theoretic reduction (Using set theory to reduce redundancy in pathway sets) prunes pathways by minimum set cover. Enrichment Map and GeneSetCluster 2.0 build networks and orderings from pairwise gene overlap. Ayllón-Benítez et al. cluster gene sets with ontology similarity measures and find that information-content measures (Resnik, Lin, Nunivers) beat edge-counting ones. We adopt their finding as our choice of hierarchy measure. gtCLIP learns joint gene-set and text embeddings with a soft gene-overlap target, and reports that over 80% of pathway pairs share no genes. All of these score pathway pairs without reference to patient-level co-variation, and none tests whether the resulting groups reproduce structurally in a new cohort. gtCLIP's cross-dataset test measures cluster quality, not whether the structure is the same.

Two factor-model papers are the closest technical neighbors. PathCORE-T maps autoencoder or matrix-factorization features to pathways and corrects for shared genes after the fact; that correction cut a TCGA network from 3826 to 119 edges. We move the overlap correction inside the covariance model instead of applying it to a fixed upstream black box. PathFA constrains loadings with flat binary Hallmark masks and is validated on a single consortium. We use the hierarchy as a soft rotation target rather than a hard mask, and we make held-out confirmation the primary result.

The psychometric scale-validation literature supplies the validation procedure. Examples include the split-sample work on the Secondary Traumatic Stress Scale—Chinese Version, which found two factors where Western samples showed three, and on AI trust (Trusting Your AI Agent Emotionally and Cognitively: Development and Validation of a Semantic Differential Scale for AI Trust). Tiong et al. show that EFA can expose hidden nuisance factors such as output-length bias in model benchmarks. That result is our motivation for expecting a "shared-gene" or "batch" factor to appear if it is not modeled.

Two papers constrain the design. CPCC regularization embeds hierarchies through edge-counting tree distance, which is exactly the class of measure Ayllón-Benítez et al. found unreliable on the Gene Ontology, and it assumes trees. We therefore do not use it. POPPY shows that ontology-structured losses can transfer across six independent melanoma trials. Its cohort collection is our main cross-dataset setting.

Research Program

The work has three parts. The first is the measurement pipeline. For each cohort, we run enrichment, keep a fixed catalog of pathways, and compute per-sample activity scores. This turns a ranked list into a patients-by-pathways matrix. Restricting attention from tens of thousands of genes to roughly 50–300 pathways also removes the problem of having far more variables than patients that breaks factor analysis on raw expression. We compute $\Omega$ from discovery-cohort gene covariance, fit regularized EFA with overlap-adjusted residuals, and rotate toward the information-content target. For small cohorts, the regularization is shrinkage of the sample correlation matrix.

The second part is the replication test. The discovery loading pattern is frozen as a confirmatory factor analysis (CFA) model, which tests whether a pre-specified factor structure fits new data. It is fit to each held-out cohort, with $\Omega$ recomputed from that cohort's own gene covariance, since the mechanical term is not a parameter of the discovery data. We report standard fit indices: the comparative fit index (CFI), root-mean-square error of approximation (RMSEA), and standardized root-mean-square residual (SRMR). We also report Tucker's congruence between independently fitted loadings, and nested invariance tests that separate "same factors" from "same loadings."

The third part is a set of competing hypotheses evaluated on the same held-out data. We fit four CFA models with the same number of factors. In the first, factors are defined by gene-overlap clusters. In the second, factors are defined by information-content hierarchy clusters. In the third, factors are defined by gtCLIP embedding clusters. In the fourth, factors come from the discovery-cohort EFA. Comparing their held-out fit answers empirically which notion of relatedness describes patient biology. We also test, with a partial Mantel statistic, whether co-loading on replicated factors tracks hierarchy similarity after controlling for overlap. The partial Mantel statistic correlates two pathway-by-pathway matrices while holding a third fixed.

Baselines

The naive reference point is the raw enrichment list with no de-replication, scored as one factor per pathway. Every reduction method must beat it on held-out fit penalized for the number of parameters, or it has thrown away information.

The overlap baselines are agglomerative clustering on Jaccard and Szymkiewicz–Simpson coefficients, as in Enrichment Map, and the seriation clustering of GeneSetCluster 2.0. Both run on the identical pathway catalog. Their cluster count is tuned on the discovery cohort over the same grid as our factor count.

The hierarchy baseline is the Lin and Nunivers information-content clustering of Ayllón-Benítez et al. with their most-specific-representative-term selection. It is our hardest conceptual competitor, because it is our rotation target used without any data.

The strongest existing method is gtCLIP. It reports 3.3-fold higher within-cluster gene-set overlap and 3.9-fold higher silhouette than its strongest baseline, and 92.4% mean normalized-enrichment-score sign coherence on five blood-cancer cohorts. On zero-shot COVID-19 data it reports 1.35x higher within-cluster overlap than GeneSetCluster 2.0. We will use released weights, if available, without fine-tuning, matching its own zero-shot protocol. Within-cluster overlap rewards precisely the mechanical correlation we subtract, so we will report it but not treat it as decisive. We will also report gtCLIP's sign-coherence metric, computed per held-out cohort.

The factor-model baselines are unconstrained EFA with Varimax and the PathCORE-T pipeline (non-negative matrix factorization features plus overlap correction). We will run PathFA where its single-modality mode applies.

The key ablations remove $\Omega$, swap the hierarchy target for Varimax, and swap the information-content target for an edge-counting one. Together these isolate which ingredient produces replication.

On the target question, no reported result was found on Lacuna for held-out CFA fit or factor congruence of any pathway-level factor model in transcriptomics. Beating the strongest competitor therefore means two things. First, held-out CFI and RMSEA must be better than both the gtCLIP-defined and hierarchy-defined models, by margins exceeding bootstrap intervals across cohorts. Second, held-out sign coherence must be no lower.

Why not just…?

Why not just cluster by gene overlap, as Enrichment Map does? Overlap clustering groups pathways by the very quantity that manufactures their correlation, so its groups will look coherent in any cohort for mechanical reasons. It is also blind to the more than 80% of pairs that share no genes, per gtCLIP. The PathCORE-T result, 3826 edges reduced to 119 after correction, suggests most overlap-driven links are artifacts. Our ablation that drops $\Omega$ tests whether overlap groups survive once mechanical correlation is removed.

Why not just use gtCLIP clusters? gtCLIP is trained to agree with gene overlap and text. Nothing in it sees which pathways co-vary in patients, and its cross-cohort test is cluster quality rather than structural reproduction. It remains a real threat. If the gtCLIP-defined CFA fits held-out cohorts as well as our EFA-derived model, then patient data add little, and we will report that a text-plus-gene embedding is a sufficient redundancy definition.

Why not just drop shared genes before scoring, then run ordinary EFA? This is the most serious simple alternative, and it may work. It loses signal for small pathways, where shared genes can make up most of the membership. Its results also depend on how shared genes are assigned. We run it as a full competitor on identical data. If it matches the $\Omega$ model on held-out fit, we will recommend it as the simpler tool.

Why not just use the ontology hierarchy and skip the data? If the hierarchy-only model replicates as well as the data-rotated one, the honest conclusion is that the Gene Ontology already encodes patient-level redundancy. That would itself be new, because it has never been tested against held-out covariance. The partial Mantel analysis and the target-versus-Varimax ablation tell us where the data depart from the ontology.

Proposed Experiments

The primary setting is the six independent melanoma immunotherapy trials (330 bulk tumors) used by POPPY. We run leave-one-trial-out discovery and confirmation. With roughly 55 samples per trial, we restrict to the 50–80 most frequently enriched Hallmark and Gene Ontology Biological Process pathways and use shrinkage estimates. The second setting is TCGA Pan-Cancer, as used by PathCORE-T. Here we discover on one set of tissue-source sites and confirm on disjoint sites within a cancer type, then test transfer across cancer types to separate site batch effects from tissue biology. If accessible, the five blood-cancer cohorts and the COVID-19 set from gtCLIP provide a head-to-head comparison on gtCLIP's own data. The Chaussabel and blood transcriptional module catalogs serve as alternative pathway vocabularies to check that conclusions do not depend on the catalog.

Three checks decide the work. The first is a simulation with known ground truth. We plant latent processes in synthetic expression using Hallmark gene sets, as in PathFA's synthetic design, and add pure-overlap pathway pairs with no shared process. The $\Omega$ model must recover the planted factors and assign the decoy pairs no common factor, while uncorrected EFA and overlap clustering must fail on the decoys. The second is a leave-one-trial-out ranking of the four competing hypothesis models on held-out CFI, RMSEA and congruence, with bootstrap intervals over patients. The claim fails if the EFA-plus-$\Omega$ model does not beat both the hierarchy-only and gtCLIP-defined models outside those intervals. The third is the invariance analysis. If factors replicate in shape but loadings differ across trials, that difference localizes cohort-specific biology or batch effects. Whether the extra factors look like batch is checked against recorded platform and site labels.