Microscopic-Mamba: Revealing the Secrets of Microscopic Images with Just 4M Parameters

Shun Zou, Zhuo Zhang, Yi Zou, Guangwei GaoarXiv 2024 · arXiv · OpenReview

Abstract

In the field of medical microscopic image classification (MIC), CNN-based and Transformer-based models have been extensively studied. However, CNNs struggle with modeling long-range dependencies, limiting their ability to fully utilize semantic information in images. Conversely, Transformers are hampered by the complexity of quadratic computations. To address these challenges, we propose a model based on the Mamba architecture: Microscopic-Mamba. Specifically, we designed the Partially Selected Feed-Forward Network (PSFFN) to replace the last linear layer of the Visual State Space Module (VSSM), enhancing Mamba's local feature extraction capabilities. Additionally, we introduced the Modulation Interaction Feature Aggregation (MIFA) module to effectively modulate and dynamically aggregate global and local features. We also incorporated a parallel VSSM mechanism to improve inter-channel information interaction while reducing the number of parameters. Extensive experiments have demonstrated that our method achieves state-of-the-art performance on five public datasets. Code is available at https://github.com/zs1314/Microscopic-Mamba

Introduction

Microscopic image classification (MIC) is a fundamental task in medical diagnostics, essential for identifying disease markers at the cellular level. Historically, researchers have faced a significant trade-off between local perception and global context. Convolutional Neural Networks (CNNs) excel at extracting fine-grained local details but suffer from a limited receptive field. In contrast, Vision Transformers (ViTs) capture long-range dependencies effectively but incur quadratic computational costs that scale poorly with image resolution.

State Space Models (SSMs), particularly the Mamba architecture, have emerged as a promising alternative, offering linear complexity. However, standard Mamba models often encounter "local pixel forgetting." Because these models process 2D images as flattened 1D sequences, pixels that are spatially adjacent in 2D can become distant in the sequence, leading to a loss of critical local structural information. To overcome this, we introduce MambaMIC, a highly efficient backbone designed specifically for the nuances of microscopic imaging.

Methodology

The core of MambaMIC is a novel block architecture that bridges the gap between local and global feature modeling through a dual-branch approach.

The MambaMIC Block

The architecture splits feature extraction into two parallel paths: a local branch and a global branch. The local branch utilizes depth-wise and point-wise convolutions to maintain spatial awareness and capture fine-grained textures. The global branch leverages the Residual Efficient Vision State Space Module (REVSSM), which employs parallel modules to mitigate the computational bottlenecks typically associated with increasing hidden states in SSMs.

The MambaMIC architecture integrates a dual-branch system for local CNN-based and global SSM-based feature extraction.

Locally Aware Enhanced Filter (LAEF)

To resolve the "forgetting" issue, we integrated the LAEF within the global branch. This module routes a specific subset of channels through a local enhancement path, re-injecting spatial information into the global sequence. Ablation studies indicate that a partial ratio of 1/4 for local enhancement provides the optimal balance between accuracy and computational speed.

Feature Modulation Interaction Aggregation (FMIAM)

Rather than simple summation, the Feature Modulation Interaction Aggregation Module (FMIAM) adaptively weights and dynamically aggregates features from both branches. By applying channel attention, the module ensures that the most relevant semantic information is prioritized before final classification.

Experimental Results

MambaMIC was evaluated across five diverse public medical datasets: RPE Data (retinal pigment epithelium), MHIST (colorectal polyps), SARS (malaria), TissueMnist (kidney cortex), and MedFM-Colon (pathology tissue).

Efficiency and Performance

MambaMIC-t (the "tiny" variant) achieves state-of-the-art results with only 4.32M parameters and 0.64 GMACs. On the RPE dataset, it reached an Overall Accuracy (OA) of 88.41%, outperforming significantly larger models like FastViT-ma36 while utilizing 90.2% fewer parameters.

Radar map demonstrating MambaMIC's superior balance of accuracy, parameter count, and computational efficiency compared to other SOTA models.

Comparison with Existing Backbones

When compared to standard Mamba-based vision models like VMamba, MambaMIC-t showed a nearly 10% accuracy improvement on the MHIST dataset. This performance gain, coupled with a reduced parameter footprint, underscores the effectiveness of the dual-branch modulation strategy in medical contexts.

Conclusion

MambaMIC establishes a new efficiency baseline for microscopic image classification. By addressing the inherent limitations of 1D scanning in SSMs through local enhancement and dynamic feature aggregation, the model achieves superior accuracy with a fraction of the computational cost of traditional CNNs or Transformers. Future work will explore the application of this architecture to more complex tasks such as microscopic image segmentation and real-time object detection.

Summary updated

Abstract

In the field of medical microscopic image classification (MIC), CNN-based and Transformer-based models have been extensively studied. However, CNNs struggle with modeling long-range dependencies, limiting their ability to fully utilize semantic information in images. Conversely, Transformers are hampered by the complexity of quadratic computations. To address these challenges, we propose a model based on the Mamba architecture: Microscopic-Mamba. Specifically, we designed the Partially Selected Feed-Forward Network (PSFFN) to replace the last linear layer of the Visual State Space Module (VSSM), enhancing Mamba's local feature extraction capabilities. Additionally, we introduced the Modulation Interaction Feature Aggregation (MIFA) module to effectively modulate and dynamically aggregate global and local features. We also incorporated a parallel VSSM mechanism to improve inter-channel information interaction while reducing the number of parameters. Extensive experiments have demonstrated that our method achieves state-of-the-art performance on five public datasets. Code is available at https://github.com/zs1314/Microscopic-Mamba
BibTeX
@misc{zou2024microscopic,
  author = {Shun Zou and Zhuo Zhang and Yi Zou and Guangwei Gao},
  title = {{Microscopic-Mamba: Revealing the Secrets of Microscopic Images with Just 4M Parameters}},
  year = {2024},
  eprint = {2409.07896},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.07896},
}
Figure