Does a Cross-Species Cell Model Think in Human?

Multilingual language models carry an internal habit that interpretability work has measured directly. They convert text in any language into a shared middle-layer representation tilted toward English, do their work there, and convert back at the output. A small set of language-specific neurons near the input and output handles the surface details of each language. Cross-species single-cell foundation models now raise the same question for biology. These are transformers trained on gene-expression profiles of individual cells from many organisms, with genes described by protein-sequence embeddings so that different genomes share one vocabulary. Their training corpora are dominated by human and mouse cells, much as language corpora are dominated by English. Nobody has checked whether they also route non-model organisms through a human-anchored internal hub, or whether that routing explains when zero-shot transfer of cell-type labels to a distant species succeeds or fails.

This note proposes to test that directly and then use the answer. We will locate species-specific units and features layer by layer inside TranscriptFormer. We will measure how well each species' cells align with homologous human cells in the middle layers. Then we will ask whether a targeted intervention on the species-specific part of the representation improves label transfer to distant species. The intervention has to beat simple embedding-level corrections; if it does not, the result is still worth reporting. The sharpest prediction is counterintuitive. If the hub is human-anchored, labelling sea-lamprey cells against a human reference should work better than labelling them against zebrafish, their closer sampled relative. That would be the biological counterpart of a model "thinking in English."

Core Claim

We claim that cross-species cell transformers show the same three-part internal layout reported for multilingual language models. Species-specific computation concentrates in early and late layers. Middle layers hold a shared cell-identity space whose geometry is set by the data-dominant species. How well a cell aligns to that dominant species in the middle layers predicts, cell by cell, whether its zero-shot label transfer succeeds. If this holds, suppressing the species-specific component at the right depth should improve transfer to distant species, and it should do so better than subtracting a global species offset from the final embedding.

The claim can fail in informative ways. The species signal might be spread across all layers with no U-shape. Alignment to human might not predict transfer once phylogenetic distance is controlled. Or a crude mean-shift correction might match any mechanism-informed intervention. Each of these outcomes tells the field something it does not currently know.

Prior Work and What It Leaves Open

The language-side evidence supplies both the hypothesis and the tools. Language-Specific Neurons introduced an entropy score over per-language neuron activation probabilities, called LAPE. It found that about 1% of neurons are language-specific, that they cluster in the bottom and top layers, and that deactivating them breaks generation in one language only. Wendler et al. and Zhao et al. describe an English-centred middle hub and a convert-reason-convert workflow. Can you map it to English? contributes two things: an instance-level alignment score (DALI), and causal evidence from activation patching in middle layers that alignment with English predicts per-example transfer success. Shared Circuits for Shared Grammar shows that circuits are shared when the underlying operation matches, regardless of genealogy. That motivates our control for phylogeny: shared function, not relatedness, may drive alignment. Understanding Multilingualism in Mixture-of-Experts LLMs shows that layer-wise steering can exploit such structure. We reuse the LAPE score, the DALI score and activation patching, adapted to cells. The language findings motivate the hypothesis, but they come from English-dominated text and may not carry over. Testing whether they do is the point.

On the biology side, TranscriptFormer is a generative model trained on 112 million cells from 12 species, using ESM-2 protein embeddings as gene tokens. It is our object of study. It reports zero-shot cell-type F1 above 0.65 on stony coral and sea lamprey, but it never inspects its own internals. SATURN integrates species through clusters of protein-embedding-similar genes ("macrogenes") and a weakly supervised triplet loss. It reaches 85.8% median frog-to-zebrafish label transfer accuracy, but it uses within-species labels and is not a transformer whose layers can be traced. Sparse Autoencoders Reveal Interpretable Features trained sparse dictionaries on single-species models. Clamping batch-associated features raised the Pancreas integration score from 0.50 (unsteered) and 0.65 (native scGPT correction) to 0.69. That work supplies our feature-level intervention, which we extend from technical batches to species. It did not trace features across depth or across organisms.

Two papers constrain the design. Zero-shot evaluation reveals limitations of single-cell foundation models shows that zero-shot scGPT and Geneformer embeddings lose to highly variable gene selection, scVI and Harmony. A Simple and Comprehensive Benchmark shows that a 0.64M-parameter MLP comes within a few points of a much larger transformer. Together they mean any gain we claim must be measured against simple corrections, not only against the unmodified model.

Research Program

The first stage is a layer-wise census of species specificity. We run TranscriptFormer's TF-Metazoa variant over the species it was trained on and compute a LAPE-style entropy for every feedforward unit, treating species as "languages." We repeat this on BatchTopK sparse-autoencoder features trained at every layer, not just one middle layer. Units and features with low entropy across species are species-specific. We record how they are spread over depth and whether species with similar protein repertoires share specific units. This is the analogue of Chinese and Japanese sharing about a quarter of their specific neurons. Because single-cell batches confound species with lab and protocol, entropy is also computed across batches within each species. A unit counts as species-specific only if it is specific to the species and not to one batch.

The second stage measures hub bias. We need "parallel sentences," meaning cells known to be the same type in two species. We take them from cell types annotated across the Tabula Sapiens, Tabula Muris and Tabula Microcebus mammalian atlas, from the frog and zebrafish embryogenesis atlases, and from the coral and lamprey sets used by TranscriptFormer. For each layer and each species pair, we compute a DALI-style score: whether a cell is closer to its homologous counterpart than to mismatched types. We then test whether each species aligns more strongly to human than to its phylogenetic neighbour. We also compare the TF-Metazoa, TF-Exemplar and TF-Sapiens variants. These differ in how much non-human data they saw, which gives a natural dose-response test of whether data dominance sets the hub.

The third stage is causal. Following the patching protocol of Can you map it to English?, we take query cells from a distant species whose label transfer fails. We replace only the activations of the species-specific features at a chosen layer with the mean activations those features take in the reference species. Then we ask whether the transferred label flips to the correct one. Sweeping the layer locates where species identity can be removed without destroying cell identity. The best layer and feature set, chosen on held-out tissues, defines a fixed intervention that we then evaluate as a zero-shot transfer method.

Baselines

The naive reference point is TranscriptFormer embeddings used unmodified. Query cells are labelled by k-nearest-neighbour voting against the reference species. On coral and lamprey this is the setting where the paper reports F1 above 0.65. Our reproduction must recover that range before any comparison counts.

The strongest existing method for the frog and zebrafish benchmark is SATURN. It reports 85.8% median label transfer accuracy and a 119% gain over SAMap on frog-to-zebrafish. We run it with its released defaults on our exact splits. Because it uses within-species labels, it effectively has more supervision than our zero-shot setting. We present matching it, rather than beating it, as success on that benchmark. For the mammalian atlas and the coral and lamprey sets, no reported result for SATURN was found on Lacuna, so we will run it ourselves.

Two simple embedding-level corrections are the real competitors to a mechanistic intervention. Global species mean-shift subtracts each species' average final-layer embedding before neighbour voting. Harmony integrates the final embeddings with species treated as a batch. Both are tuned on the same held-out tissues and with the same search budget as our layer and feature choice. We also include scVI on 1:1 orthologs, following the evidence that classical tools beat zero-shot foundation-model embeddings.

Two ablations isolate the key ingredient. The first applies our intervention to randomly chosen sparse-autoencoder features matched in number and sparsity. The second applies it at a randomly chosen layer. Beating the baselines means a higher median per-cell-type accuracy and macro F1 on held-out species pairs than both mean-shift and Harmony, with confidence intervals from cell-type bootstrap, while not lowering the integration score that measures how well biological clusters are preserved.

Why not just…?

Why not just subtract the species mean from the final embedding? This is the most serious threat. If the species signal is roughly a constant offset, a global shift captures it and the mechanistic work adds only explanation. The layer sweep settles it. If species identity is cleanly removable only at a specific depth, and is entangled with cell type by the final layer, the targeted intervention should win on cell types whose species signature varies, such as lineage-restricted types. If mean-shift ties it everywhere, we will report that the hub is effectively linear and stop at the descriptive result.

Why not just use SATURN? SATURN already beats orthology-based methods. But it needs within-species annotations and offers no view of how a large pretrained model routes species internally. Our question concerns zero-shot transfer to unannotated organisms like lamprey, where those labels do not exist. On frog and zebrafish we run SATURN head-to-head, and we will say plainly if our zero-shot intervention falls short of its 85.8% median.

Why not just fine-tune TranscriptFormer on the target species or use orthologs with scVI? Fine-tuning needs target labels, which defeats the zero-shot setting. The ortholog route discards more than half the genes for distant species, according to SATURN. We include scVI on orthologs anyway, because the zero-shot evaluation paper shows such tools often beat foundation-model embeddings. Losing to it on close mammalian pairs would be unsurprising. Losing on coral and lamprey would count against the proposal.

Decisive Checks

Three results decide the proposal. The first is the pivot test. Lamprey and coral query cells are labelled against a human reference and against the nearest sampled non-human species. Matched cell types and matched reference sizes are enforced by subsampling. A human-anchored hub predicts that the human reference wins in TF-Metazoa and wins by more in TF-Exemplar. Phylogenetic proximity predicts the reverse. The second is predictive validity. Middle-layer DALI score must predict per-cell transfer success after controlling for phylogenetic distance and reference-set size, mirroring the 30-of-33 pattern reported for languages. The third is the intervention test. The targeted species-feature replacement must flip failing transfers more often than random-feature and random-layer ablations, and must beat mean-shift and Harmony on held-out species pairs.

One caveat is the single-cell analogue of translationese. Annotations in non-model organisms are often derived from human or mouse markers, which could inflate apparent alignment to human. We will therefore report the pivot test separately on cell types whose labels in the query species were defined from independent markers, where the source annotation allows this.