TL;DR
CellPainTR is a Transformer-based model that learns reusable, batch-invariant representations of Cell Painting morphological profiles. By replacing standard self-attention with efficient Bidirectional Hyena operators and conditioning on learnable source-context tokens, CellPainTR effectively disentangles technical batch artifacts from true biological signals. Remarkably, it not only outperforms standard tools on in-distribution data across 13 laboratory sources, but also generalizes zero-shot to an unseen external dataset under severe feature dropout.
What Problem Does This Paper Address?
Cell Painting is a high-content fluorescent microscopy assay that extracts thousands of morphological features (cell shape, intensity, texture) across multiple cellular compartments, widely used in functional genomics and drug discovery. However, the data is plagued by severe batch effects—systematic, non-biological variations introduced by differences in laboratories, instruments, reagent lots, and plate layouts.
Existing batch correction tools (such as ComBat and Harmony) operate as static, one-time correctors on a fixed dataset. When a new batch or laboratory dataset arrives, the entire combined dataset must be re-processed and re-aligned from scratch. This fundamental lack of generalization prevents the scientific community from building a cumulative, federated "atlas" of cellular morphology where new screening data can be mapped directly onto a stable reference.
Methods
To overcome the quadratic complexity bottleneck of standard Transformers over thousands of continuous morphological features, the authors designed CellPainTR:
- Feature Embedding & Linear Adaptor: Rather than binning into discrete tokens, continuous feature values are mapped to a 256-dimensional embedding space via a linear adaptor and combined with a learnable feature context embedding (replacing standard positional encodings, since morphological features are non-sequential).
- Bidirectional Hyena Operators: The architecture replaces standard self-attention with 4 stacked Bidirectional Hyena layers. With near-linear computational complexity, Hyena enables unrestricted modeling of long-range dependencies across thousands of morphological features simultaneously without truncation or downsampling.
- Source-Context Token (
[SRC]): A learnable embedding token drawn from a provenance codebook is prepended to the input sequence alongside a[CLS]token, explicitly conditioning the network on technical origin. - Three-Stage Training Curriculum:
- Channel-Wise Masked Morphology (CWMM): Self-supervised reconstruction of masked feature values grouped by imaging channel and compartment (e.g., DNA–nucleus, Mito–cytoplasm).
- Intra-Source Fine-Tuning: Combines CWMM with supervised contrastive learning (using compound InChIKey labels) exclusively within individual sources to remove plate-to-plate noise.
- Inter-Source Generalization: Trains contrastive loss across batches with mixed laboratory sources, forcing the model to map identical compounds from different sources into the same biological representation.
Datasets
- In-Distribution (JUMP Consortium
cpg-0016): Over 100,000 compound and genetic perturbations generated across 13 different partner laboratories, using CellProfiler to produce 4,765 morphological features per profile. - Out-of-Distribution Stress Test (Bray et al., 2016): An external dataset originating from an entirely unseen laboratory and an older CellProfiler pipeline (yielding only 1,383 features). Only 275 common features overlapped with the JUMP feature schema; the remaining input positions were simply zero-padded.
Key Insights and Ideas
- Pre-trained representations can transcend dataset boundaries: Classical batch correctors must be re-fit on target data. CellPainTR demonstrates that a model trained on diverse experimental sources learns an invariant biological manifold capable of zero-shot inference on external data.
- Batch invariance versus biological signal is an explicit trade-off: The authors' ablation study revealed that purely self-supervised masking excels at batch removal but captures less semantic biological nuance, while early supervised contrastive learning over-indexes on biological identity. The final inter-source curriculum successfully navigates this Pareto frontier.
- Context-aware conditioning works via proxy: When evaluating completely unseen datasets where no learned source token exists, assigning the token of the most experimentally similar training source ('Source 10') provides an effective conditioning proxy.
Key Results
- In-Distribution (JUMP held-out test): CellPainTR achieved the highest Overall aggregate score (0.60), outperforming ComBat (0.54), Harmony (0.56), and uncorrected baseline (0.56). It achieved an aggregate Batch Correction score of 0.76 and Biological Preservation score of 0.45, striking the best global balance between removing source noise and separating Mechanisms of Action (MoA).
- Out-of-Distribution (Bray et al.): Despite receiving only 275 of its 4,765 expected features (over 94% feature dropout) and zero fine-tuning, CellPainTR scored 0.40 Overall (Batch Correction: 0.66, Bio Metrics: 0.15). In contrast, ComBat (0.26) and Harmony (0.25)—even when re-fit directly on the target data—performed no better than the uncorrected baseline (0.26).
Limitations and Future Work
- Manual Source Proxy Selection: Applying the model to out-of-distribution datasets currently requires manually matching experimental metadata to choose a proxy
[SRC]token; future work should learn to infer or dynamically generate source tokens directly from sample statistics. - Reliance on Feature Extractors: CellPainTR operates on tabular CellProfiler features rather than end-to-end on raw microscope images, inheriting any potential segmentation or feature extraction biases.
- Benchmark Scope: The current evaluation compares against classical batch correction algorithms (ComBat, Harmony, Sphering); future comparisons against recent deep autoencoder architectures would provide deeper benchmarking context.
Summary updated