Co-Detection-Aware Memory Banking for Single-Cell Encoders on FPGAs

Deep models for single-cell RNA sequencing (measurements of which genes are active in each individual cell) spend much of their compute in one place. That place is the first layer, which reads a vector of 20,000–30,000 gene counts per cell, over 90% of them zero. Skipping the zeros is the obvious fix, and scGPT already does it in software. On a parallel hardware accelerator, though, skipping zeros turns a dense matrix multiply into a gather: for each detected gene, fetch its column of weights. Which genes are detected changes from cell to cell, so parallel fetch units collide on the same memory bank. Designers then add wide crossbars and arbiters to resolve the collisions, and on FPGAs (reconfigurable chips whose speed is limited by programmable wiring) those crossbars are where routing congestion caps the clock.

Our observation is that, although the set of detected genes varies per cell, it is statistically predictable. Genes that are detected together in a reference atlas will be detected together in new cells. We propose to assign genes to memory banks once, offline, so that genes likely to co-occur land in different banks. Narrow, conflict-light memory paths could then replace wide crossbars, with no per-cell preprocessing. We pair this with fixed, balanced sparsity in the gene-to-hidden weights. The aim is to measure, using real place-and-route outcomes rather than FLOP counts, how much clock speed and energy this buys and what it costs in biological quality.

Core Claim

For a lightweight single-cell encoder such as scVI, we make three claims. A gene-to-bank assignment derived from atlas co-detection statistics will cut bank conflicts per cell well below random or address-hashed assignment. The reduction will carry over to held-out tissues and batches. And it will let the gather engine use fewer crossbar ports, yielding higher post-route maximum clock frequency and lower energy per cell at matched throughput, with latent-space quality statistically indistinguishable from the dense model.

The assignment problem has a clean form. Let $b(g)\in\{1,\dots,B\}$ be the bank of gene $g$, and let $p_{gh}$ be the fraction of atlas cells in which genes $g$ and $h$ are both detected. The expected number of same-bank collisions per cell is

$$C(b)=\sum_{g<h} p_{gh}\,\mathbb{1}[b(g)=b(h)],$$

and we minimize $C(b)$ under a balance constraint that each bank holds roughly $G/B$ genes, where $G$ is the number of genes. This is a balanced weighted graph partitioning problem on the co-detection graph.

Positioning Against Prior Work

The method borrows from two hardware papers and departs from each at a specific point. Pre-Defined Sparse Neural Networks with Hardware Acceleration fixes a balanced connection pattern before training and builds a clash-free banking schedule. That guarantee holds only because the inputs are dense, so the access pattern is fully determined by the weights. Here the access pattern also depends on which inputs are nonzero, so a clash-free guarantee is impossible and must be replaced by an expected-conflict objective. Serpens colors nonzeros to avoid bank conflicts in sparse matrix-vector products. The authors themselves note that this per-matrix preprocessing prevents use on matrices that change at runtime. Our assignment is per gene, not per matrix, so it is computed once and applied to streaming query cells.

Banked Memories for Soft SIMT Processors supplies the hardware toolkit: carry-chain arbiters and an "Offset" address mapping that improved throughput about 10% over naive mapping. Their mapping is workload-agnostic, whereas ours uses data statistics. Flexible Communication Avoiding Matrix Multiplication on FPGA with High-Level Synthesis motivates the work. It documents a ceiling near 80–90% logic utilization, beyond which routing congestion blocks timing closure. We target exactly the regime where gather crossbars push designs past that ceiling.

On the biology side, Assessing the limits of zero-shot foundation models in single-cell biology constrains the choice of model. It shows that scVI beats Geneformer on clustering and that simple highly-variable-gene selection beats both on batch integration. That result is why we accelerate a small VAE rather than a foundation model. scGPT showed that omitting zero-expressed genes cuts fine-tuning time by up to 60% on GPUs, the software analogue of our gather. We ask what that same skip costs on a fixed parallel datapath. scvi-tools supplies the model, the training code, and the scArches reference-mapping workflow we use as the target task.

Research Program

The work has three parts. The first is offline. We compute co-detection statistics on a training split of a reference atlas and solve the balanced partition with a multilevel graph partitioner. We then train scVI encoders whose first layer has pre-defined balanced sparsity: each hidden unit reads a fixed number of genes, with an equal number drawn from every bank. This keeps per-bank load even.

The second part is a cycle-level model of a banked gather engine. Given a cell's detected genes, a bank assignment, a bank count $B$, and a port width, the model counts stall cycles. We use it to sweep the design space cheaply.

The third part is hardware. A small number of selected configurations are implemented in high-level synthesis and taken through full place-and-route on an HBM-equipped FPGA of the kind Serpens used. The engine streams query cells in compressed sparse form, gathers weight columns from banked on-chip memory, accumulates hidden activations, and finishes the encoder in a small dense stage. We report post-route maximum clock frequency, logic and routing utilization, cells per second, and measured energy per cell. Crossbar width is the knob that trades conflict tolerance against wiring, and we sweep it explicitly.

Baselines

The naive reference point has two parts. In software it is the standard dense scVI encoder in scvi-tools on a GPU, with no sparsity. In hardware it is the same gather engine with genes assigned to banks by low address bits. That mapping is what an engineer gets by default, and it shows the conflict rate and crossbar cost that the problem naturally carries.

The strongest hardware competitor is the gather engine with Offset address mapping and carry-chain arbitration from Banked Memories for Soft SIMT Processors. That paper reports 771 MHz and 60.4% read bank efficiency on matrix transposition, but those are not scRNA workloads; we found no result on Lacuna for any FPGA accelerator on single-cell data. We will implement this baseline in the same HLS code base, on the same board, with the same bank count and the same place-and-route effort and seeds. Beating it means one of two things on held-out tissues: at least 1.3× lower conflict stalls per cell at an equal crossbar width, or equal stalls with half the crossbar ports and a measurably higher post-route clock.

The strongest software competitor is scVI with zero-skipping on a modern GPU, timed end to end on the same query sets. On quality, the reference number is scVI's median average silhouette width of 0.54 on Tabula Sapiens, from Assessing the limits of zero-shot foundation models in single-cell biology. On speed, scvi-tools reports that scArches mapped a query in 10 minutes, an 8× speedup over retraining, from scvi-tools. We will reproduce both rather than rely on the reported figures. The GPU gets the same tuning budget as the FPGA. Our success criterion against it is energy per mapped cell; we do not expect to win on raw throughput, and we will say so if we lose on energy too.

Two ablations isolate the key ingredient. In the first, we keep the pre-defined balanced sparsity but use random balanced bank assignment. This separates the benefit of the co-detection objective from the benefit of sparsity itself. In the second, we build the partition from bulk expression correlation instead of single-cell co-detection. This tests whether the dropout-specific co-occurrence structure is what matters.

Why not just…?

Why not just run scVI on a GPU? For atlas-scale batch work this is a real threat. scvi-tools already scales linearly to roughly 500,000 cells, and our design will likely not beat a data-center GPU on throughput. The case for this work rests on two narrower claims, both of which we test directly. One is energy per cell at low power, which matters for instrument-side or continuous mapping. The other is the general question of whether data-dependent sparse gathers can avoid the routing cliff. Measured joules per cell against the GPU settle the first; if the FPGA does not win there, we report the hardware result as a characterization study and not as a deployment argument.

Why not just select 2,000 highly variable genes and shrink the input layer? This is the most serious alternative. A 10× smaller gene vocabulary may make the weight table small enough to replicate across banks, removing conflicts without any clever assignment. Highly-variable-gene pipelines are also strong on integration (Assessing the limits of zero-shot foundation models in single-cell biology). We therefore run every hardware and quality experiment at both 2,000 genes and the full gene set. If full-gene models give no quality gain in cross-tissue reference mapping, and the engine with 2,000 genes and replication fits within the routing budget, we will conclude that the partitioning matters only for full-vocabulary models.

Why not just use a workload-agnostic mapping with more banks? Adding banks lowers conflicts for any mapping. However, each added bank widens the lane-to-bank crossbar, and that crossbar is precisely the structure documented as the timing bottleneck, both in Banked Memories and in Serpens' place-and-route failure when scaling from 16 to 24 HBM channels. Our sweep of bank count against post-route clock frequency measures this trade directly. If Offset mapping with 2× banks closes timing at the same frequency as our design, the alternative wins and we will report that.

Why not just accelerate scGPT? The zero-shot evaluation shows that scGPT's representations are frequently no better than scVI or highly-variable-gene baselines. A larger model would add cost without a demonstrated quality benefit for reference mapping.

Proposed Experiments

We use Tabula Sapiens (483k cells) and the Immune Cell Atlas (330k) as reference atlases, with tissue-held-out and batch-held-out query splits. PBMC 12k/95k, Pancreas, and the 18-batch COVID-19 dataset serve as external queries. Quality is measured by average silhouette width, the batch integration score, and label-transfer accuracy after scArches mapping. The quality test is non-inferiority against dense scVI, within bootstrap confidence intervals.

The first decisive check is generalization of the conflict reduction. Is the reduction in $C(b)$ measured on held-out tissues and new sequencing batches at least half of the reduction seen in-distribution? If co-detection structure does not transfer, the central idea fails.

The second decisive check is a hardware curve. Across bank counts and crossbar widths, we plot post-route clock frequency and utilization against conflict stalls for our assignment, Offset mapping, and random assignment. Our assignment must reach a better operating point than both, not merely a lower conflict count.

The third check is the 2,000-gene versus full-gene comparison described above, which decides whether the method's scope covers all single-cell encoders or only full-vocabulary ones.