ISG: I can See Your Gene Expression

Yan Yang, Liyuan Pan, Liu Liu, Eric A. Stone

TL;DR

This paper introduces ISG, a novel framework for predicting gene expression from histology slide images. ISG addresses the challenge of high-resolution, sparsely textured images by intelligently filtering non-informative regions and then leveraging a dual attention mechanism to capture interactions between relevant local and global image features, leading to state-of-the-art performance.

What Problem Does This Paper Address?

Predicting gene expression from histology slide images is crucial for understanding diseases and developing treatments. However, histology images present significant challenges: they are extremely high-resolution (e.g., 10^5 x 10^5 pixels), making traditional end-to-end deep learning approaches computationally prohibitive. Furthermore, these images often contain large areas of textureless or uniformly colored regions, interspersed with areas rich in discriminative textures. Existing methods often struggle with this heterogeneity, either by failing to filter out these textureless regions or by aggregating features uniformly, thus neglecting important spatial interactions and long-range dependencies within the image. This leads to suboptimal gene expression prediction.

Methods

The proposed ISG framework tackles these challenges with three novel modules:

  1. Shannon Selection Module: This module filters out textureless or featureless image patches. Instead of relying on traditional edge detection, it uses information theory, specifically Shannon information content and Solomonoff's universal prior, to quantify the "texture abundance" of each patch. Patches with low information content (indicating lack of texture) are discarded, focusing subsequent processing on more informative regions.
  2. Feature Extraction Network: This module extracts low-dimensional feature representations from both "global" (coarse-resolution) and "local" (fine-resolution) patches. It employs a style-based architecture, trained in an unsupervised manner using an image reconstruction task. This approach is chosen for its ability to capture versatile feature representations, which is essential given the limited availability of labeled gene expression data.
  3. Dual Attention Network: This is the core of ISG's interaction mechanism. It takes the global and local features extracted in the previous step and refines them. The network has two components:
  4. Local Feature Updater: Uses global features to guide the updating of local features, emphasizing regions of interest.
  5. Global Feature Updater: Uses local features to update global features, reflecting the evolving significance of local details. This dual attention mechanism allows for adaptive calibration of model attention to relevant regions, effectively integrating contextual information and capturing complex interactions between different parts of the histology image.

Datasets

The primary dataset used for evaluation is the TCGA dataset, which contains histology slide images and corresponding gene expression data for six different cancer types. The authors also validate their method on a separate dataset for microsatellite instability (MSI) status prediction, demonstrating its generalization ability.

Key Insights and Ideas

  • Information Theory for Patch Selection: Using information content to filter image patches is a novel approach that effectively addresses the heterogeneity of histology images, moving beyond traditional image processing techniques.
  • Unsupervised Feature Extraction for Gene Expression: Leveraging style-based architectures for unsupervised feature learning is crucial for gene expression prediction, given the scarcity of paired image-gene expression data.
  • Dual Attention for Spatial Interactions: The ISG framework's dual attention mechanism is key to capturing the complex spatial relationships between local and global image features, which are critical for accurate gene expression prediction.
  • Generalizability Beyond Gene Prediction: The successful application of ISG to MSI status prediction highlights its potential for broader applications in computational pathology.

Key Results

  • ISG significantly outperforms existing state-of-the-art methods, including HE2RNA, on the TCGA dataset for gene expression prediction, achieving the best Pearson Correlation Coefficient (PCC) across various gene types and cancer types.
  • The Shannon Selection module, when integrated into HE2RNA, improves its performance by 7.5%, demonstrating the module's effectiveness in filtering irrelevant patches.
  • In the MSI status prediction task, ISG achieves competitive AUC scores compared to specialized methods.
  • The proposed Feature Extraction network demonstrates superior performance compared to a pretrained ResNet50 when used for gene expression prediction, highlighting the quality of its learned representations.
  • ISG is more computationally efficient than HE2RNA, with faster inference times.

Limitations and Future Work

  • Computational Cost: While ISG is more efficient than HE2RNA, the high resolution of histology images still poses computational challenges.
  • Threshold Selection: The paper investigates the impact of the Shannon Selection threshold, suggesting that an optimal threshold is crucial for performance. Further research could explore more adaptive thresholding strategies.
  • Model Architecture: The paper ablates the number of MBCONV blocks and attention fusion blocks in the Dual Attention network, identifying an optimal configuration (ISG-10-2) for PCC. However, there's no strict correlation between PCC and L2 loss, suggesting that different metrics might favor different architectural choices. Future work could explore architectures that better balance these objectives.

Summary updated

BibTeX
@inproceedings{yang2022isg,
  author = {Yan Yang and Liyuan Pan and Liu Liu and Eric A. Stone},
  title = {{ISG: I can See Your Gene Expression}},
  booktitle = {BMVC},
  year = {2022},
}

Loading related content...

Figure