Learning Sparse Interpretable Features For NAS Scoring From Liver Biopsy Images

Chong Yin, Siqi Liu, Vincent Wai-Sun Wong, Pong C. YuenIJCAI 2022 · DOI

TL;DR

This paper introduces SparseX, a novel deep learning method for automatically scoring Non-alcoholic Fatty Liver Disease (NAFLD) activity from liver biopsy images. By intelligently selecting informative tissue regions and learning sparse, interpretable features, SparseX achieves competitive diagnostic performance while offering better insights into the underlying pathology.

What Problem Does This Paper Address?

Non-alcoholic fatty liver disease (NAFLD) is a growing global health concern, and liver biopsy remains a crucial diagnostic tool. The NAFLD Activity Score (NAS) quantifies the severity of histological findings like steatosis, inflammation, and ballooning. However, analyzing these biopsy images is challenging due to the complex tissue structures and the sparse, localized nature of the pathological findings within the vast whole-slide images (WSIs). Traditional methods often struggle to efficiently pinpoint these crucial areas, leading to potential inaccuracies and a lack of interpretability. Furthermore, obtaining detailed pixel-level annotations for training deep learning models is labor-intensive and often unavailable. This paper aims to develop an automated system that can accurately score NAS while being interpretable and efficient, even with limited annotations.

Methods

The proposed method, SparseX, tackles the problem in two main stages:

  1. Histological Feature-guided Interpretable Spatial Sampling (ISS): This module addresses the spatial sparsity of pathological findings. It first uses a pre-trained histological feature extractor to identify key tissue structures (like nuclei and fat droplets) that are indicative of alterations. These features then guide a "scorer" network to estimate an "attention score" for each image tile within the WSI, highlighting regions likely to contain pathological findings. The TopK selection strategy is then employed to select the most informative tiles based on these attention scores, effectively focusing the subsequent analysis on relevant areas.
  2. NMF-based Attributes Learning (SAL): This module addresses feature sparsity. The selected informative tiles are fed into a deep network to extract high-dimensional features. To learn sparse and interpretable features, Non-negative Matrix Factorization (NMF) is integrated into the deep network. NMF decomposes the feature representation into low-rank matrices, allowing the model to learn "core attributes" that represent dominant tissue information. The method then uses a classifier on these core attributes to predict the NAS. The overall objective function combines the NMF loss with a classification loss, guiding the decomposition to discover attributes most relevant for diagnosis.

Datasets

The authors evaluated their method on two liver biopsy image datasets:

  • SteatosisRaw: A public dataset of liver biopsy images collected from 33 children, stained and normalized to H&E.
  • Liver-NAS: A private dataset of liver biopsy images from 265 patients, stained with H&E without normalization processing.

Both datasets provide NAS scores assigned by expert pathologists for steatosis, inflammation, and ballooning. The authors also mention combining some classes in the original NAS distribution due to small sample sizes for certain scores.

Key Insights and Ideas

  • Histological features as guides: The core idea of using readily identifiable histological features (nuclei, fat droplets) to guide the selection of informative image tiles is a clever way to overcome the challenge of sparse pathological findings without requiring explicit pixel-level annotations of the findings themselves.
  • Sparsity in two domains: The paper explicitly addresses both spatial sparsity (pathological findings are localized) and feature sparsity (only a subset of learned features are relevant for diagnosis), proposing distinct but complementary solutions for each.
  • NMF for interpretability: Integrating NMF into a deep learning framework allows for the learning of interpretable "core attributes," which can potentially offer more insight into why a certain NAS score is assigned, beyond just a prediction.
  • Adaptive sampling: The ISS module allows the model to adaptively focus on different regions of the WSI based on the presence of histological cues, making it more efficient than processing the entire WSI uniformly.

Key Results

The proposed SparseX method, particularly when combining ISS and SAL (ISS+SAL), demonstrated strong performance across both datasets.

  • On the SteatosisRaw dataset, ISS+SAL achieved an average F1 score of 73.1% for steatosis, 76.7% for inflammation, and 74.7% for ballooning.
  • On the Liver-NAS dataset, ISS+SAL achieved an average F1 score of 73.4% for steatosis, 56.8% for inflammation, and 70.8% for ballooning.
  • The paper highlights that ISS alone improved performance over baseline methods like "All-samples" and "TopK," and the addition of SAL further boosted results, especially for inflammation and ballooning, which are noted as being more sparse.
  • Qualitative results (attention maps) showed that SparseX's attention distribution more accurately reflected the spatial distribution of histological findings compared to the TopK method, indicating improved interpretability.

Limitations and Future Work

  • Dataset size and staining variance: The authors note that on the more challenging Liver-NAS dataset, which has staining variance and is smaller, the performance gains from SAL were less pronounced. This suggests that SAL might be more effective with larger datasets or when staining is more consistent.
  • Feature compression ratio (r): The paper mentions that there isn't a simple linear relationship between the feature compression ratio (r) and performance, and that feature sparsity has a greater impact on the smaller SteatosisRaw dataset. This indicates that finding the optimal sparsity level for features might require further investigation.
  • Future work: The authors suggest that future work could explore how to leverage the different degrees of sparsity observed for different histological findings to further improve model performance. They also imply that the interpretability offered by the core attributes could be further explored.

Summary updated

BibTeX
@inproceedings{yin2022learning,
  author = {Chong Yin and Siqi Liu and Vincent Wai-Sun Wong and Pong C. Yuen},
  title = {{Learning Sparse Interpretable Features For NAS Scoring From Liver Biopsy Images}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/220},
}
Figure