Introduction
Spatial transcriptomics (ST) has revolutionized our understanding of tissue architecture by mapping gene expression within its physical context. A critical task in ST analysis is the identification of "spatial domains"—functionally distinct regions such as brain layers or tumor microenvironments. However, existing computational methods often struggle to effectively integrate the multimodal nature of ST data, which includes high-dimensional gene expression, spatial coordinates, and high-resolution histological images. Many current approaches either ignore image data or use simplistic fusion techniques that fail to capture the complex interdependencies between a cell's genetic profile and its morphology. Furthermore, standard contrastive learning techniques often mistakenly penalize biologically similar cells as "negatives," degrading model performance.
Methodology
To address these challenges, we present IE-HERCL (Image-Enhanced Hybrid Encoding with Reinforced Contrastive Learning). The framework utilizes a sophisticated multi-stage pipeline designed for robust multimodal integration:
- Hybrid Dual-Encoding: The model employs Autoencoders (AE) to capture non-spatial features and GraphSAGE encoders to model spatial dependencies. This dual-encoding is applied independently to both gene expression data and image features (pre-extracted via ResNet50).
- Cross-Modal Triple-Attention: Rather than simple concatenation, IE-HERCL uses a triple-attention mechanism to adaptively fuse gene and image representations, ensuring the final latent embedding is coherent and biologically informative.
- Reinforced Contrastive Learning: To solve the "false negative" problem, the authors implement high-order random walks on an affinity graph. This allows the model to identify and group biologically similar samples even if they are not identical views, preventing the model from being penalized for grouping similar cells together.
- Optimal Transport (OT) Refinement: The model uses Sinkhorn divergence to dynamically align the learned clustering distribution with an auxiliary distribution, effectively "sharpening" the boundaries of identified spatial domains.
Experimental Results
IE-HERCL was evaluated across five diverse spatial transcriptomics datasets, including NanoString Lung 9-1 and various brain and cancer tissues. The model achieved state-of-the-art (SOTA) performance across all benchmarks, measured by the Adjusted Rand Index (ARI) and Normalized Mutual Information (NMI).
Key findings from the experimental evaluation include:
- Superior Accuracy: On the Human Breast Cancer dataset, IE-HERCL outperformed the second-best method (stDCL) by 4.66% in ARI.
- Robustness: The model demonstrated consistent performance in complex tissue structures where traditional methods like Scanpy or STAGATE produced fragmented or noisy results.
- Ablation Success: Studies confirmed that the inclusion of image data and the reinforced contrastive learning strategy are essential components, as removing either significantly degraded the quality of domain identification.
Conclusion
IE-HERCL represents a significant advancement in spatial domain identification by effectively leveraging histological images and accounting for biological similarity within a contrastive learning framework. While the computational complexity of random walks and Optimal Transport is a factor for very large datasets, the model's ability to produce clean, biologically plausible tissue maps makes it a powerful tool for spatial biology. Future work may involve integrating additional modalities like spatial proteomics and utilizing domain-specific pre-trained image models to further enhance performance.
Summary updated