Which Layer to Share: Federated Encoders with Site-Local Calibration for Real-Time Cell Sorting

Machines that image cells as they flow past and physically sort them within microseconds now exist. One example is an event-camera sorter that classifies red blood cells in about 395 μs and sorts 1,000 cells per second. Each such device, though, learns alone. Each lab trains its own classifier on its own cells, and the images that would make a better shared model, such as patient-derived blood, rare tumor cells, or proprietary compound screens, usually cannot leave the institution. The standard answer is federated learning: train one model across sites by exchanging parameter updates instead of data. A closed-loop sorter adds a complication, because its decision has to be made faster than any network round trip. It also meets site-specific shifts in optics, staining, and cell handling that a single shared model handles poorly.

This note proposes to treat the sorter's model as a stack of layers and to ask empirically which layers should be shared. At the bottom is an image encoder, the network that turns a cell image into a feature vector. It is trained jointly across sites, slowly and asynchronously, and never on the real-time path. Above it sit two thin site-local layers. The first is a normalization that corrects that site's batch effects. The second is a decision policy, meaning the per-site threshold or small classifier head that actually triggers the sorting actuator. The claim is that this split recovers nearly all of the benefit of pooled training. It also beats both isolated local training and a fully shared federated model. And it keeps the deployed loop inside a hard latency budget, because federation only swaps in a new frozen encoder between runs. We also expect the benefit of federating to be large only for imaging modalities that lack a big public training corpus, and we design the study so it can show federation is unnecessary where one exists.

Core Claim

Write the deployed predictor at site $s$ as

$$\hat{y} = \pi_s\big(\nu_s(g_\theta(x))\big),$$

where $x$ is a single-cell image, $g_\theta$ is the encoder with parameters $\theta$ shared across sites, $\nu_s$ is a site-specific feature normalization (for example per-feature centering and scaling or a small affine map fit on local control cells), and $\pi_s$ is the site's decision layer (a threshold or linear head). Only $\theta$ is federated. The testable hypothesis concerns where the split sits. Federating $\nu_s$ as well should hurt under batch shift. Federating $\pi_s$ as well should hurt when sites differ in prevalence or in the cost of errors. Federating nothing should hurt small sites.

The idea of keeping the decision layer local comes from clinical prediction. Optimizing the Implementation of Clinical Predictive Models to Minimize National Costs: Sepsis Case Study separated a sepsis risk score from a department-level threshold policy. It reports projected savings of $4.6 billion, $700 million better than a uniform 80% sensitivity threshold. That study was retrospective, used data from one health system, and was not federated. We carry its prediction/policy split into a federated, real-time setting, and we add the middle normalization layer that the morphology literature says is necessary. The motivation for that middle layer is the batch problem that Cell Painting Gallery: an open resource for image-based profiling describes as requiring continual re-normalization across illumination, staining, and laboratories.

Relation to Prior Work

The closest systems each cover part of this and leave out the rest. Neuromorphic-enabled video-activated cell sorting supplies the real-time regime and the latency target. It pairs an event camera with a spiking network on a neuromorphic chip and reports 0.99% average error on spherocyte versus normal red cell sorting, compared with 19.93% from single frames. It trains offline on one device and has no way to update across sites. CODA: an open-source platform for federated analysis and machine learning on distributed healthcare data supplies the federated layered architecture: site nodes behind firewalls, a coordinating hub, and user applications. It shows that federated mortality prediction matches pooled training (AUC 0.82 versus 0.81). It is built for batch retrospective analysis of health records and radiographs, has no streaming path, and has no morphology engine. Privacy for Personal Neuroinformatics keeps raw signals local and releases only low-dimensional answers. We borrow its principle that raw images and the local decision never leave the site, but we add collaborative training of the encoder, which that architecture does not support.

On the morphology side, three papers constrain the design. Three million images and morphological profiles of cells treated with matched chemical and genetic perturbations (CPJUMP1) reports that only 5–25% of compounds match their sister compounds and that compound-to-gene matching is barely above chance. So any federation gain has to be measured against a weak, noisy retrieval signal, with replicate-level controls. Predicting cell health phenotypes using image-based morphology profiling shows that per-readout Elastic Net models on Cell Painting features predict S-phase fraction and percent dead cells with $R^2$ near or above 0.8 in some cell lines, but fail on finer events. This gives a natural shared-features, local-heads task. Label-free cell cycle analysis for high-throughput imaging flow cytometry shows that 213 brightfield and darkfield features predict DNA content with $r = 0.896$ in fixed and $r = 0.786$ in live Jurkat cells, all post-acquisition. Label-free imaging is what a live sorter would use. It is also the modality with no large public corpus, which is where we expect federation to matter.

We found no study in the file that federates training across multi-site high-content morphological data. The Cell Painting Gallery is a central store. We therefore report no prior federated morphology number and use pooled training as the reference upper bound.

Research Program

The work has three parts.

The first is a controlled federated benchmark built from existing data, with "sites" defined by real acquisition differences rather than random splits. In CPJUMP1 we treat the four combinations of cell line (U2OS, A549) and time point as sites, and we also split by plate or batch where the metadata allows. In the cell health data the three cell lines (A549, ES2, HCC44) are sites. In the label-free cytometry data, fixed and live Jurkat acquisitions form two sites with a strong built-in shift. To make one site much smaller than the rest, which is the realistic case for a small clinical lab, we subsample it.

The second part is the training method itself. Encoders are convolutional or small vision transformers trained on single-cell crops. A self-supervised objective is shared across sites, and supervised heads stay local. Encoder updates are aggregated with federated averaging, and secure aggregation is added in a later arm. Each site fits its normalization $\nu_s$ on its own negative-control cells and its decision layer $\pi_s$ on its own labeled cells, with a threshold chosen for that site's error costs.

The third part is a latency study. We compile the frozen local stack to embedded hardware and measure per-cell inference against the 395 μs figure from the event-camera sorter. We also inject federation rounds, including slow sites, during streaming to confirm that inference never waits on them.

Baselines

The naive reference point is local-only training. Each site trains the full encoder, normalization, and head on its own data, with the same architecture, the same number of optimizer steps per site, and the same hyperparameter search as every other arm. For the label-free task this reference already has a reported number from engineered features: DNA-content correlation $r = 0.896$ fixed and $r = 0.786$ live in Label-free cell cycle analysis for high-throughput imaging flow cytometry. We rerun that feature pipeline with boosting inside each of our sites so it becomes a like-for-like arm. The proposal beats this reference if the smallest site gains clearly over its own local model, which is where federation should help most.

The strongest existing model-level baseline is the engineered-feature approach of Predicting cell health phenotypes using image-based morphology profiling: CellProfiler features plus per-readout Elastic Net, reported $R^2$ near or above 0.8 for S-phase and dead-cell fraction in some lines. Fed its own features, it needs no deep encoder and no federation. It is strong because hand-designed features transfer across sites for free. We run it with the same train/test compound splits and report per-readout $R^2$. Our method beats it only if it does better on the readouts where Elastic Net is weak, not merely on the easy ones.

The strongest federated competitor is full federated averaging, where encoder, normalization, and head are all shared. This is how a CODA-style platform would train by default. It gets the same communication rounds and compute. The split method beats it if it matches or exceeds it on average and clearly exceeds it on the most shifted site.

Pooled centralized training is the upper bound. It shows how much of the pooling benefit each federated arm recovers.

Two ablations isolate the key ingredient, the placement of the split. One additionally federates the normalization layer. The other additionally federates the decision layer. The metrics are CPJUMP1 replicate and sister-compound retrieval, measured as the fraction correctly matched (the metric CPJUMP1 reports, with 5–25% for sister compounds), cell health $R^2$, label-free DNA-content correlation and phase accuracy, and per-cell latency.

Why not just…?

Why not just pool the extracted feature profiles centrally? Profiles are small, so the bandwidth argument drawn from the 656 TB image archive does not apply to them. If a consortium can legally pool, it should, and pooled training is our upper bound. The case for federation depends entirely on governance: patient-derived cells and proprietary screens that cannot move. The CODA deployment across eight hospitals is the evidence in the file that such constraints are real. Our experiments do not argue for federation where pooling is allowed. They measure how close federation gets when pooling is ruled out.

Why not just pretrain on public Cell Painting data and recalibrate locally, with no federation at all? This is the most serious threat, and we include it as a full experimental arm. The encoder is trained on public gallery data and only $\nu_s$ and $\pi_s$ are fit locally. For stained Cell Painting tasks we expect this arm to come close to federation, because the gallery is large and matches those tasks. The deciding test is label-free brightfield and darkfield data, for which no comparable public corpus exists in the file. If public pretraining plus local recalibration matches federation there too, we will report that federation is unnecessary for sorters and drop it from the recommended design.

Why not just meta-analyze locally trained models? Secure and federated genome-wide association studies for biobank-scale datasets reports that meta-analysis lost power and diverged from joint analysis across heterogeneous cohorts. Its setting is statistical association testing, not representation learning, so the result may not carry over. Our local-only arm, plus a variant that ensembles or averages local predictions, tests this directly on morphology.

Why not just run one monolithic tightly coupled model on the device? The deployed stack is already a single frozen artifact. The layers exist at training time, not as runtime middleware. The latency study checks this claim: if the split adds measurable per-cell overhead compared with a single fused network, that counts against the design.

Proposed Experiments and Decisive Checks

The first decisive check is the split ablation on the most shifted pair, fixed versus live label-free Jurkat, and on the smallest CPJUMP1 site. The hypothesis survives only if keeping $\nu_s$ and $\pi_s$ local beats sharing them. The difference must exceed seed-to-seed variation across five seeds.

The second check is the public-pretraining arm on label-free data, described above. It is the kill criterion for federation.

The third check is whether the split method recovers most of the gap between local-only and pooled training at the smallest site. We report the fraction of the gap recovered, not only absolute scores.

The fourth check is latency. The local stack must stay under 395 μs per cell on sorter-class hardware while federation rounds with slow sites run in the background. That figure comes from event-camera inference, and the sorters it applies to are the target deployment.

Finally, because shared encoder updates can leak information about which perturbations a site screened, a concern raised in the file for federated summaries generally, we run a membership-inference probe on the encoder updates. We then repeat the main comparison with secure aggregation to measure its cost in accuracy and time.