Gene co-expression analysis groups thousands of genes into "modules" of genes whose activity rises and falls together across patients. These modules are then linked to disease. The most widely used method, weighted gene co-expression network analysis (WGCNA), builds a gene-by-gene similarity matrix, clusters it with a greedy tree-building procedure, and reports a single partition of genes into modules. Two weaknesses compound each other. The similarity WGCNA trusts most, topological overlap (how many strong neighbors two genes share), is an $N \times N$ matrix that is expensive to build and store when $N$ covers the full transcriptome. The greedy clustering then makes merge decisions it can never undo, so the one partition it reports may be an arbitrary member of a large family of nearly equally good partitions.
We propose a GPU engine that computes topological overlap in tiles, uses each tile immediately, and throws it away, so the full matrix is never stored. Its main job is to score an entire population of candidate partitions in one streamed pass. That cost structure makes it affordable to run an island-model evolutionary search over module partitions: several semi-isolated populations evolve separately and occasionally exchange their best members. The search returns a diverse set of high-scoring module maps instead of one tree cut. The scientific claim is testable. Modules that persist across that diverse set should replicate in independent patient cohorts better than modules from the standard tree cut, and genes whose membership flips across the set should be the ones whose module assignment fails to replicate.
Core Claim
WGCNA as described in Identifying Functional Gene Modules via Co-Expression Networks proceeds in four steps. It turns absolute gene–gene correlations $s_{ij}$ into adjacencies $A_{ij} = s_{ij}^\beta$. It computes topological overlap
$$\text{TOM}_{ij} = \frac{(A^2)_{ij} + A_{ij}}{\min(k_i, k_j) + 1 - A_{ij}},$$
where $k_i = \sum_u A_{iu}$ is gene $i$'s total connection strength. It clusters $1-\text{TOM}$ hierarchically. Finally, it cuts the resulting tree into modules. Applied studies (RNF181 in coronary artery disease; breast cancer hub genes CENPL, ISG20L2, LSM4, MRPL3) use this pipeline as is. The coronary artery disease study is the motivating evidence for our gap. It names two limitations. First, it had to restrict the analysis to 8,525 genes, possibly dropping low-expression regulators. Second, it notes that hierarchical clustering is irreversible.
Our observation is that topological overlap is a matrix multiply followed by an elementwise correction that uses only per-gene vectors ($k$) and the adjacency entry itself. That shape fits a fused GPU kernel. The relevant prior work is FlashSinkhorn (FlashSinkhorn), which fuses cost computation and reductions in on-chip memory for optimal transport. It states that it supports only costs that decompose into dot products. TOM's $\min(k_i,k_j)$ denominator does not decompose that way, but it is cheap to apply in the kernel's final step once each tile of $A^2$ exists.
The second observation matters more. Every consumer of TOM we need reduces it: top-$k$ neighbor graphs, intramodular connectivity, and block sums over pairs of modules. Those reductions can be accumulated tile by tile, for many partitions at once. The new contribution is therefore not "TOM on GPU." It is turning TOM into a streamed objective that a diversity-preserving population search can afford to query every generation. The search output is then judged by cross-cohort replication rather than by its own score.
Research Program
Streaming kernel. The engine stores the standardized expression matrix ($N$ genes by $n$ samples, small) and the adjacency $A$ in half precision. At $N = 60{,}000$ that is roughly 7 GB, which fits a 24 GB consumer GPU. It never stores the full TOM. Each output tile is computed as a tensor-core matrix multiply of row blocks of $A$, finished with the TOM correction, and handed to reducers in the same kernel.
The reducers produce three outputs:
- an exact top-$k$ TOM neighbor list per gene;
- connectivity within each module;
- a population score accumulator.
The population score accumulator works as follows. For a population of $P$ partitions with up to $K$ modules each, encoded as a one-hot membership matrix $M$ of size $N \times PK$, the engine accumulates the block sums $M^\top \text{TOM}\, M$. This is a second tensor-core multiply per tile. Its cost is comparable to the TOM multiply itself when $PK$ is on the order of $N$, so scoring 256 partitions costs about one extra TOM pass rather than 256.
We will also test full recomputation of $A$ from the expression matrix inside the kernel. The rough cost model is an overhead factor of about $2n/T$ for tile size $T$, which comes from recomputing each $A$ tile once per reuse. This is affordable only for small cohorts, and we will report where it pays off.
Search. A partition's score is a modularity-like contrast computed from the block sums. Let $S_{ab}$ be the summed TOM between modules $a$ and $b$, and $n_a$ the size of module $a$. The score is
$$Q(P) = \sum_a \Big( \tfrac{S_{aa}}{n_a^2} - \tfrac{\sum_{b \ne a} S_{ab}}{n_a (N - n_a)} \Big) - \lambda K.$$
Here $\lambda$ penalizes module count, and singleton "grey" genes are excluded as in WGCNA.
Islands are seeded from different starting points: the standard dynamic tree cut, Leiden community detection on the top-$k$ TOM graph, and random merges of fine-grained clusters. Mutation operators reassign genes on module boundaries, split modules, and merge modules. Periodic migration of elites between islands follows the PySR island design (Review of PySR).
A known weakness of novelty search, documented in NS-ES, is the need for a hand-designed description of behavior. We avoid it because partitions have a natural distance, the variation of information between two partitions. Each island keeps an archive of partitions within a tolerance of the best score and rewards distance from that archive. The output is the archive: a set of near-optimal, mutually different module maps. From it we compute a per-gene membership-stability score.
Runtime constraint. The interpreted-language direction plays a supporting role. Following the PySR lesson that cross-language glue and multithreading can cost speed and break reproducibility, the whole generation loop runs on the GPU, called from a thin Python front end. Reductions use a fixed order so that results are bitwise reproducible across runs. This matters for biomarker claims, and the PySR review reports that seeding alone fails to guarantee it.
Baselines
The naive reference point is standard R WGCNA with dynamic tree cut. It runs on the gene subset that fits memory, roughly the top 8,000–10,000 variable genes, mirroring the coronary artery disease study, and it is also run on the full gene set where hardware allows. Any partition we propose must beat it on cross-cohort replication, not on our own score $Q$, which tree cut never optimizes.
The strongest systems competitor is dense GPU TOM: plain half-precision matrix multiply in PyTorch on an 80 GB A100, materializing the full TOM. At single-cohort scale this is what more memory buys, and it is the honest ceiling for our kernel's speed. We run it on identical inputs and precision and report wall-clock time and peak memory for $N$ from 5,000 to 60,000. We also report the maximum absolute TOM error of both methods against a float64 reference.
The strongest clustering competitor is Leiden community detection on the exact top-$k$ TOM graph produced by our own kernel. It optimizes a modularity-type objective with moves that can be undone, which directly answers the irreversibility complaint. It gets the same graph and a resolution sweep with as many configurations as the evolutionary search has generations, under the same GPU-hour budget.
A third baseline is bootstrap consensus clustering: resample patients, re-run tree cut, and cluster the co-assignment matrix. It also yields per-gene stability, but stability with respect to sampling noise rather than degeneracy of the objective.
The key ablation removes diversity preservation. A single population of the same total size with no islands, no novelty term, and no migration runs under the same budget. A (1+1) hill climber is run as well, since Understanding the Importance of Evolutionary Search found such greedy search competitive early but stalling on harder combinatorial problems.
No published number exists for any of these baselines on the replication metrics we use. We found no reported result on Lacuna for module-preservation statistics, GO enrichment, or WGCNA runtime on TCGA-BRCA, METABRIC, or the coronary artery disease GEO series. We will generate these numbers ourselves under the matched protocol.
The only quantitative anchors in the file belong to the applied studies and are used as sanity checks:
- tree-cut WGCNA found 7 modules over 8,525 genes on the coronary artery disease data (RNF181 paper);
- single-gene diagnostic AUC in TCGA was 0.948 for LSM4 and 0.934 for CENPL (breast cancer hub genes). We should rediscover these genes as high-connectivity hubs in stable modules.
Beating a baseline means three things on held-out cohorts:
- higher median module preservation in the second cohort;
- a higher fraction of modules with replicated trait association;
- membership stability that predicts which genes fail to replicate, measured by AUC above 0.5 with a bootstrap confidence interval excluding it.
Why not just…?
Why not just store the full TOM on a large-memory GPU or node? For a single cohort at $N \le 40{,}000$ on an 80 GB A100 this works. Dense matrix multiply is compute-bound, so the bandwidth argument behind FlashSinkhorn's reported 161× end-to-end speedup over KeOps (FlashSinkhorn) does not carry over to TOM. We concede that point. The streaming kernel earns its place in three situations: on commodity 24 GB devices, at full-transcriptome $N$, and in the search loop, where many partitions must be scored per generation. Disk memory-mapping a 14 GB float32 matrix and reading it every generation is the case where streaming should clearly win. The scaling experiment reports the crossover explicitly. If dense storage is never slower at any tested $N$ on the large device, we will say so.
Why not just run Leiden on a sparse TOM graph? This is the most serious threat to the search component. Leiden already escapes the irreversible merges of tree cutting and is far cheaper than a population search. If Leiden partitions replicate across cohorts as well as the stable core of the evolutionary archive, the evolutionary claim fails. The contribution would then shrink to "exact streamed top-$k$ TOM plus Leiden," which is still useful at full-transcriptome scale. What Leiden cannot give is a map of equally good alternatives. Our decisive test is whether archive-derived membership stability predicts replication failure better than Leiden's stability across resolutions and random seeds.
Why not just use bootstrap consensus clustering for stability? Consensus clustering measures sensitivity to which patients were sampled. Our archive measures how flat the objective is at fixed data. These can disagree. A gene can sit firmly in a module under every resample yet lie on a ridge between two equally good partitions. We run both and test which stability score better predicts cross-cohort replication. We also test whether combining them helps, which would mean both are needed.
Why not just filter to the top variable genes? Filtering is cheap and standard. However, the coronary artery disease study itself lists it as a limitation, and filtering changes every gene's connectivity $k_i$ and therefore every TOM value. We test this directly by comparing modules and hub rankings for the retained genes with and without the filtered-out genes present. We report how many hub calls change and whether added low-variance genes join replicating modules.
Proposed Experiments
Synthetic data. We simulate expression with planted modules, including deliberately ambiguous genes loaded on two modules, at $N$ from 5,000 to 60,000 and $n$ from 100 to 1,000. Partitions are scored against ground truth with the adjusted Rand index. This test asks whether the archive's unstable genes are the planted ambiguous ones.
Breast cancer. We build modules on TCGA-BRCA and test preservation, trait association, and survival association in METABRIC, then repeat with the roles reversed.
Coronary artery disease. We use leave-one-dataset-out across GSE12504, GSE22253, GSE77263, GSE20681, and GSE49925 (RNF181 paper). Each run is done on the filtered and on the full gene set, and we check whether RNF181 remains a stable hub.
Systems measurements. We report peak memory, wall-clock time per TOM pass, time per generation as a function of population size, numerical error against a float64 reference, and bitwise reproducibility across ten repeated runs.
Decisive checks. Two results decide the scientific claim. The first is whether stable-core modules from the island archive replicate across cohorts more often than tree-cut modules, Leiden modules, and single-population evolutionary modules at equal compute. The second is whether per-gene archive stability predicts replication failure beyond what bootstrap consensus explains. If either fails, the diversity-preserving search is not worth its cost for co-expression analysis. We will report that outcome alongside the streaming kernel's standalone value.
References
- Session Introduction
- Weighted Co-Expression Network Analysis Identifies RNF181 as a Causal Gene of Coronary Artery Disease
- CENPL, ISG20L2, LSM4, MRPL3 are four novel hub genes and may serve as diagnostic and prognostic markers in breast cancer
- FlashSinkhorn: IO-Aware Entropic Optimal Transport on GPU
- Review of PySR: high-performance symbolic regression in Python and Julia
- Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents
- Understanding the Importance of Evolutionary Search in Automated Heuristic Design with Large Language Models