Divide and Conquer Self-Supervised Learning for High-Content Imaging

Lucas Farndale, Paul Henderson, Edward W. Roberts, Ke Yuan

Abstract

Self-supervised representation learning methods often fail to learn subtle or complex features, which can be dominated by simpler patterns which are much easier to learn. This limitation is particularly problematic in applications to science and engineering, as complex features can be critical for discovery and analysis. To address this, we introduce Split Component Embedding Registration (SpliCER), a novel architecture which splits the image into sections and distils information from each section to guide the model to learn more subtle and complex features without compromising on simpler features. SpliCER is compatible with any self-supervised loss function and can be integrated into existing methods without modification. The primary contributions of this work are as follows: i) we demonstrate that existing self-supervised methods can learn shortcut solutions when simple and complex features are both present; ii) we introduce a novel self-supervised training method, SpliCER, to overcome the limitations of existing methods, and achieve significant downstream performance improvements; iii) we demonstrate the effectiveness of SpliCER in cutting-edge medical and geospatial imaging settings. SpliCER offers a powerful new tool for representation learning, enabling models to uncover complex features which could be overlooked by other methods.

Introduction

Self-supervised learning (SSL) has become a cornerstone of modern computer vision, yet it remains susceptible to "simplicity bias." Models frequently gravitate toward the most obvious patterns—such as background textures or coarse shapes—to satisfy training objectives, often at the expense of more subtle, informative features. This "shortcut" learning is particularly detrimental in High-Content Imaging (HCI), including multiplex medical scans and hyperspectral satellite data, where the most scientifically relevant signals are often the most complex and easily overlooked.

The SpliCER Framework

To combat simplicity bias, we propose Split Component Embedding Registration (SpliCER). SpliCER is an architectural wrapper designed to force models to learn representations from every constituent part of an image.

SpliCER Architecture: The model deconstructs a composite image into components, which are then mapped to specific chunks of a global embedding.

The SpliCER pipeline consists of three primary stages:

  1. Deconstruction: The input image is partitioned into components, such as individual color channels in multiplex imaging or spatial regions defined by segmentation masks.
  2. Chunked Embedding: A primary encoder processes the full image to produce a global embedding, which is then subdivided into discrete "chunks."
  3. Registration: Each individual component is processed by a paired encoder. The resulting component embedding is then "registered" or aligned via a loss function to its corresponding chunk within the primary global embedding.

Overcoming Gradient Interference

A key insight of this work is the "Opposing Gradients" trap. In standard joint-embedding architectures, if a feature is present in only one view or channel, the gradients from other "empty" views can contradict the learning of that feature. By isolating components into specific chunks, SpliCER ensures that unique information in one part of the image is not drowned out by noise or lack of information in another.

Conceptual comparison: Standard shared embeddings can lead to opposing gradients for unshared features, whereas SpliCER's chunked approach supports learning all features.

Experimental Evaluation

SpliCER was evaluated across a variety of challenging domains:

  • Synthetic Benchmarks: Using an MNIST-CIFAR hybrid task, we demonstrated that SpliCER can learn complex CIFAR features even when simple MNIST "shortcuts" are available. SpliCER achieved 68.4% accuracy on complex features where standard baselines failed.
  • Medical Imaging: On the Orion-CRC dataset (19-channel immunofluorescence), SpliCER improved T-cell subtype classification from ~56% to over 81%. It also showed superior robustness to distribution shifts in histopathology tasks (Camelyon17).
  • Geospatial Analysis: In hyperspectral satellite imagery (EuroSAT), SpliCER improved land-use classification from 85% to 90% by better utilizing non-RGB spectral bands.

Conclusion

SpliCER provides a robust framework for ensuring that SSL models capture the full depth of information available in high-content images. By leveraging prior knowledge through image deconstruction and preventing gradient interference via chunked embeddings, SpliCER enables the discovery of complex features essential for scientific advancement. While dependent on the quality of initial deconstruction, its compatibility with existing SSL loss functions makes it a versatile tool for the next generation of representation learning.

Summary updated

Abstract

Self-supervised representation learning methods often fail to learn subtle or complex features, which can be dominated by simpler patterns which are much easier to learn. This limitation is particularly problematic in applications to science and engineering, as complex features can be critical for discovery and analysis. To address this, we introduce Split Component Embedding Registration (SpliCER), a novel architecture which splits the image into sections and distils information from each section to guide the model to learn more subtle and complex features without compromising on simpler features. SpliCER is compatible with any self-supervised loss function and can be integrated into existing methods without modification. The primary contributions of this work are as follows: i) we demonstrate that existing self-supervised methods can learn shortcut solutions when simple and complex features are both present; ii) we introduce a novel self-supervised training method, SpliCER, to overcome the limitations of existing methods, and achieve significant downstream performance improvements; iii) we demonstrate the effectiveness of SpliCER in cutting-edge medical and geospatial imaging settings. SpliCER offers a powerful new tool for representation learning, enabling models to uncover complex features which could be overlooked by other methods.
BibTeX
@misc{farndale2025divide,
  author = {Lucas Farndale and Paul Henderson and Edward W. Roberts and Ke Yuan},
  title = {{Divide and Conquer Self-Supervised Learning for High-Content Imaging}},
  year = {2025},
  eprint = {2503.07444},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.07444},
}
Figure