Bio-Inspired Network for Diagnosing Liver Steatosis in Ultrasound Images

Yuan Yao, Zhenguang Zhang, Bo Peng, Jin TangBioengineering 2023 · DOI

TL;DR

This paper introduces BiNet, a novel deep learning network inspired by the human visual cortex, for diagnosing liver steatosis from ultrasound images. By mimicking the selective attention and feedback mechanisms of the visual system, BiNet achieves high accuracy in a challenging diagnostic task, offering a more interpretable and potentially more robust approach than traditional methods.

What Problem Does This Paper Address?

Diagnosing liver steatosis, a condition where fat accumulates excessively in the liver, is crucial for preventing severe liver diseases like cirrhosis and cancer. Ultrasound is a common diagnostic tool due to its non-invasiveness and accessibility. However, ultrasound images often suffer from low quality, noise, and blur, making accurate diagnosis difficult and subjective, relying heavily on clinician experience. Existing automated methods, while improving, still face challenges in achieving consistently high performance, especially with varying image quality. This paper aims to address these limitations by developing a more sophisticated and biologically inspired deep learning model for improved accuracy and reliability in liver steatosis diagnosis.

Methods

The core innovation of this paper lies in the Bio-inspired network (BiNet), which draws inspiration from the human visual cortex's ventral pathway. BiNet incorporates several key components:

  • Receptive Field Feature Extraction Module (RFFE): This module simulates the inhibitory effect of the non-classical receptive field (nCRF) on the classical receptive field (CRF) in the primary visual cortex (V1). This mechanism helps suppress background noise and irrelevant textures, enhancing the extraction of important features.
  • Parallel Attention Module (PA Block): Instead of processing features sequentially, this module uses two parallel attention paths (WMSA and SWMSA, inspired by Swin Transformer) to process the same input simultaneously. The outputs from these paths are then fused, allowing for a more comprehensive integration of different feature information.
  • Feedback Connection: The network simulates the feedback regulation mechanism observed in higher visual cortical regions to lower ones, contributing to a more refined feature extraction process.

The overall architecture is structured in stages, mimicking the progression of visual information through V1, V2, and V4 regions of the visual cortex, culminating in a classification block that corresponds to the IT (inferotemporal) cortex for object recognition.

Datasets

The authors utilized two datasets for their research:

  • Byra et al. dataset: A publicly available dataset comprising 550 images from 55 participants.
  • Newly constructed dataset: A self-built dataset collected from elderly medical examination patients (over 65 years old) between 2020 and 2022. This dataset contains 250 ultrasound images, with 100 normal liver images and 150 images of moderately severe fatty livers. The images are 3-channel RGB with a resolution of 720x480. Data augmentation techniques like random scaling, flipping, and rotation were applied to the training and validation sets.

Key Insights and Ideas

  • Biomimicry for Robustness: The paper strongly advocates for drawing inspiration from biological vision systems to overcome limitations in current deep learning models, particularly in challenging domains like medical image analysis where noise and variability are prevalent.
  • Selective Attention as a Feature Enhancer: The selective attention mechanisms of the visual cortex are not just for recognition but also for efficient feature extraction by focusing on relevant information and ignoring noise. BiNet leverages this to improve diagnostic accuracy.
  • Parallel Processing for Richer Features: The parallel attention module's ability to process information through distinct pathways and then fuse them suggests a more holistic feature representation compared to sequential processing.
  • Interpretability through Biological Analogy: By mapping network components to specific visual cortex regions (V1, V2, V4, IT), the BiNet architecture offers a degree of interpretability, making it easier to understand why certain features are being extracted and processed.

Key Results

The BiNet model demonstrated strong performance on the developed dataset:

  • Optimal Parameters: The model achieved its best performance with 10 training epochs and a learning rate of 0.0001.
  • Accuracy: The best results reported for BiNet on the private dataset were an accuracy of 98.0%, sensitivity of 100.0%, and specificity of 96.0%, with an F1-score of 0.980.
  • Comparison with Other Models: When compared to other existing methods in Table 4, BiNet consistently outperformed them across accuracy, sensitivity, and specificity metrics, achieving an accuracy of 99.1% on the publicly available dataset.
  • Effectiveness of Parallel Attention: Ablation studies showed that the full BiNet with parallel attention blocks outperformed models using only one attention path (WMSA or SWMSA), indicating the benefit of the fused parallel processing.

Limitations and Future Work

The authors acknowledge several limitations:

  • Focus on Ventral Pathway: The current BiNet primarily models the ventral pathway of the visual system. The full visual processing involves more complex pathways and initial stages (e.g., photoreceptors) that are not yet incorporated.
  • Limited Datasets: The model was trained and tested on only two datasets. The scalability and generalizability of BiNet might be limited without exposure to a wider variety of data.

Future work suggested includes:

  • Incorporating More Biological Mechanisms: Integrating additional effective biological vision mechanisms into deep learning models to further enhance performance.
  • Expanding Datasets: Training and testing on more diverse datasets to improve the model's robustness and generalizability.
  • Improving Classification Performance: The authors believe that with more datasets, the classification performance can be further improved.

Summary updated

Abstract

Using ultrasound imaging to diagnose liver steatosis is of great significance for preventing diseases such as cirrhosis and liver cancer. Accurate diagnosis under conditions of low quality, noise and poor resolutions is still a challenging task. Physiological studies have shown that the visual cortex of the biological visual system has selective attention neural mechanisms and feedback regulation of high features to low features. When processing visual information, these cortical regions selectively focus on more sensitive information and ignore unimportant details, which can effectively extract important features from visual information. Inspired by this, we propose a new diagnostic network for hepatic steatosis. In order to simulate the selection mechanism and feedback regulation of the visual cortex in the ventral pathway, it consists of a receptive field feature extraction module, parallel attention module and feedback connection. The receptive field feature extraction module corresponds to the inhibition of the non-classical receptive field of V1 neurons on the classical receptive field. It processes the input image to suppress the unimportant background texture. Two types of attention are adopted in the parallel attention module to process the same visual information and extract different important features for fusion, which improves the overall performance of the model. In addition, we construct a new dataset of fatty liver ultrasound images and validate the proposed model on this dataset. The experimental results show that the network has good performance in terms of sensitivity, specificity and accuracy for the diagnosis of fatty liver disease.
BibTeX
@inproceedings{yao2023bio,
  author = {Yuan Yao and Zhenguang Zhang and Bo Peng and Jin Tang},
  title = {{Bio-Inspired Network for Diagnosing Liver Steatosis in Ultrasound Images}},
  booktitle = {Bioengineering},
  year = {2023},
  doi = {10.3390/bioengineering10070768},
}
Figure