Out of Distribution Generalization via Interventional Style Transfer in Single-Cell Microscopy

Wolfgang M. Pernice, Michael Doron, Alex Quach, Aditya Pratapa, Sultan Kenjeyev, Nicholas De Veaux, Michio Hirano, Juan C. Caicedo

Abstract

Real-world deployment of computer vision systems, including in the discovery processes of biomedical research, requires causal representations that are invariant to contextual nuisances and generalize to new data. Leveraging the internal replicate structure of two novel single-cell fluorescent microscopy datasets, we propose generally applicable tests to assess the extent to which models learn causal representations across increasingly challenging levels of OOD-generalization. We show that despite seemingly strong performance as assessed by other established metrics, both naive and contemporary baselines designed to ward against confounding, collapse to random on these tests. We introduce a new method, Interventional Style Transfer (IST), that substantially improves OOD generalization by generating interventional training distributions in which spurious correlations between biological causes and nuisances are mitigated. We publish our code and datasets.

Introduction: The Challenge of Batch Effects

In biomedical research, deep learning models are notorious for "cheating" by latching onto batch effects—technical noise from specific lab equipment, reagents, or experimental dates—rather than learning underlying biology. This leads to a failure in Out-of-Distribution (OOD) generalization. An "Observational Environment" (OE) in microscopy encompasses everything from the multi-well plate used to the specific operator. Models frequently learn spurious correlations, such as associating a disease phenotype with the background lighting of a particular batch.

Standard evaluation methods (IID) often fail to detect this because test data typically shares the same biases as training data. Even state-of-the-art models can collapse when tested on a new "series" or "plate," highlighting a critical gap in model reliability for scientific discovery.

Interventional Style Transfer (IST)

To address this, we propose Interventional Style Transfer (IST). IST is a generative approach designed to "impute" missing data by "hallucinating" how cells would appear if moved between different experimental environments.

Conceptual framework of Interventional Style Transfer (IST) compared to standard training.

The method involves:

  1. Style Swapping: IST combines a "content" image (biological signal) with a "style" image (environmental noise) to generate a new synthetic image.
  2. Architecture: Utilizing a ResNet-18 encoder and a UNet-decoder, IST employs skip-connections and Adaptive Instance Normalization (AdaIN) to inject style codes derived from feature map statistics.
  3. Loss Functions: A suite of losses—including adversarial, cycle-consistency, content, and a novel class-matching loss—ensures that biological morphology is preserved while the environmental "style" is altered.

By randomly pairing biological causes with different environments, IST creates an "interventional" distribution $P(X, Y | do(C))$, effectively breaking the causal link between the environment ($C$) and the biological label ($Y$).

Experimental Benchmarks

We evaluated IST using two primary benchmarks:

  • GRID (Genetics of Rare Inherited Disease): 17,030 images from 19 patients with rare mitochondrial or neuromuscular diseases, featuring a hierarchical replicate structure (wells, plates, series).
  • LINCS-SC (Library of Integrated Network-based Cellular Signatures): 200,000 single-cell images involving 1,327 compounds. This dataset is "almost perfectly sparse," as most drug treatments appear in only one specific environment.

Key Results

The impact of IST on OOD generalization is significant. On the GRID dataset, baseline models collapsed when moving from Level-1 (same plate) to Level-3 (new experimental series), with F1 scores dropping from ~0.55 to ~0.12. In contrast, IST maintained much higher performance, achieving an F1 of 0.3673 at Level-3—nearly triple the baseline performance.

Performance comparison of IST against baselines on GRID and LINCS-SC datasets.

Visual evidence from UMAP visualizations further confirms these results: while baseline models cluster images by technical "Batch," IST-trained models successfully cluster images by "Biological Cause," regardless of the laboratory or plate of origin. Furthermore, projecting test images into the "familiar" style space of the training environments was found to further enhance performance.

Discussion and Future Work

Our findings suggest that high performance on standard validation sets is a poor proxy for real-world utility in microscopy. By treating "style" as a technical confounder and applying causal interventions, we can effectively de-bias training data.

However, IST currently requires biological labels during generator training. Future research could explore self-supervised learning to remove this dependency. Additionally, while IST's pixel-level content preservation is vital for microscopy, different architectural priors may be needed for natural images where style changes involve geometric shifts like 3D rotations.

Summary updated

Abstract

Real-world deployment of computer vision systems, including in the discovery processes of biomedical research, requires causal representations that are invariant to contextual nuisances and generalize to new data. Leveraging the internal replicate structure of two novel single-cell fluorescent microscopy datasets, we propose generally applicable tests to assess the extent to which models learn causal representations across increasingly challenging levels of OOD-generalization. We show that despite seemingly strong performance as assessed by other established metrics, both naive and contemporary baselines designed to ward against confounding, collapse to random on these tests. We introduce a new method, Interventional Style Transfer (IST), that substantially improves OOD generalization by generating interventional training distributions in which spurious correlations between biological causes and nuisances are mitigated. We publish our code 1 and datasets 2 .
BibTeX
@inproceedings{pernice2023out,
  author = {Wolfgang M. Pernice and Michael Doron and Alex Quach and Aditya Pratapa and Sultan Kenjeyev and Nicholas De Veaux and Michio Hirano and Juan C. Caicedo},
  title = {{Out of Distribution Generalization via Interventional Style Transfer in Single-Cell Microscopy}},
  booktitle = {Workshops @ CVPR},
  year = {2023},
  url = {https://openreview.net/forum?id=B3797HMA8e},
}
Figure