Abstract
Large-scale biological discovery requires integrating massive, heterogeneous datasets like those from the JUMP Cell Painting consortium, but technical batch effects and a lack of generalizable models remain critical roadblocks. To address this, we introduce CellPainTR, a Transformer-based architecture designed to learn foundational representations of cellular morphology that are robust to batch effects. Unlike traditional methods that require retraining on new data, CellPainTR's design, featuring source-specific context tokens, allows for effective out-of-distribution (OOD) generalization to entirely unseen datasets without fine-tuning. We validate CellPainTR on the large-scale JUMP dataset, where it outperforms established methods like ComBat and Harmony in both batch integration and biological signal preservation. Critically, we demonstrate its robustness through a challenging OOD task on the unseen Bray et al. dataset, where it maintains high performance despite significant domain and feature shifts. Our work represents a significant step towards creating truly foundational models for image-based profiling, enabling more reliable and scalable cross-study biological analysis.
Introduction
In drug discovery and biological research, Cell Painting serves as a high-content imaging assay that captures a vast array of morphological features—such as shape, texture, and size—across multiple cellular compartments. While powerful, these datasets are highly susceptible to "batch effects": technical variations arising from differences in microscopes, reagents, or experimental timing that can obscure genuine biological signals.
Current batch correction methods, such as ComBat or Harmony, are typically "static." They are optimized for a specific, fixed dataset and often fail when applied to new data from different laboratories. This lack of generalizability hinders the creation of a unified "cellular atlas" where data from disparate sources can be integrated seamlessly. CellPainTR addresses this by moving toward a foundational, reusable deep-learning architecture.
Methods
CellPainTR utilizes a Transformer-based architecture optimized for the high-dimensional nature of morphological data. The workflow, from data preparation to multi-step training, is illustrated in Figure 1.
Architectural Innovations
- Hyena Operators: To manage the thousands of features in Cell Painting data without the quadratic computational cost of standard Transformers, CellPainTR employs Hyena operators. These provide near-linear complexity, enabling the model to capture long-range dependencies across the entire feature space efficiently.
- Source Context Tokens ([SRC]): The model incorporates a learnable "source codebook." By using specific source tokens, the model can explicitly account for the technical noise associated with a particular lab or experiment.
Training Curriculum
The model is trained through a three-stage process:
- Channel-Wise Masked Morphology (CWMM): A self-supervised task where the model learns to predict missing feature values, establishing a baseline understanding of morphological "grammar."
- Intra-Source Fine-tuning: Employs supervised contrastive learning to ensure that cells with identical biological treatments exhibit similar representations within a single source.
- Inter-Source Generalization: The model is trained on mixed batches from multiple sources to develop a globally consistent, source-invariant representation.
Data Evaluation
The model was primarily trained and tested on the JUMP Cell Painting Consortium dataset (>100,000 perturbations). To test Out-of-Distribution (OOD) generalization, the authors used the Bray et al. (2017) dataset, which was entirely unseen during training and contained significantly fewer features (275 vs. 4,765), requiring the model to handle extreme feature mismatch.
Results
CellPainTR demonstrates superior performance in both in-distribution and out-of-distribution scenarios. In the JUMP dataset, it achieved an overall score of 0.60, surpassing established benchmarks like ComBat (0.54) and Harmony (0.56).
A key finding was the model's robustness to feature mismatch. Despite the Bray et al. dataset having 94% fewer features, CellPainTR maintained high performance by zero-padding missing dimensions. This suggests the model learns high-level biological concepts rather than simple feature correlations. For OOD tasks, the researchers utilized a "proxy" source token from the codebook based on metadata similarity, which outperformed methods specifically retrained on the new data.
Discussion and Future Work
While CellPainTR represents a significant advancement, it currently relies on experimental metadata to select proxy source tokens for new datasets. Future work aims to automate this process by inferring source characteristics directly from the data. Additionally, while the current model operates on engineered features (e.g., from CellProfiler), future iterations could transition to an end-to-end approach, learning directly from raw pixel data. By establishing a foundational model for cell morphology, CellPainTR paves the way for more scalable and reliable cross-study biological analysis.
Summary updated