High-Accuracy Image Segmentation Based on Hybrid Attention Mechanism for Sandstone Analysis

Lanfang Dong, Hao Gui, Xiaolu Yu, Xinming Zhang, Mingyang XuMinerals 2024 · DOI

TL;DR

This paper introduces a novel Mask R-CNN-based model for high-accuracy sandstone image segmentation, employing a hybrid attention mechanism and a shape-aware loss function to overcome challenges like grain adhesion and irregular shapes. The authors also present a new, high-quality dataset (SMISD) to facilitate better training and evaluation, demonstrating superior performance on this dataset compared to existing methods.

What Problem Does This Paper Address?

Accurate mineral identification and textural characterization are crucial in geological exploration, particularly for sandstones, which often contain valuable hydrocarbon resources. Traditional methods rely on manual microscopy, which is labor-intensive, subjective, and time-consuming. While computer vision and deep learning have advanced image segmentation, existing methods struggle with the specific challenges in sandstone analysis:

  • Adhered and concealed grains: Leaching processes can cause grains to stick together or obscure their boundaries, making precise segmentation difficult.
  • Small and irregular grains: Segmenting very small or irregularly shaped grains remains a significant challenge for current algorithms.
  • Complex backgrounds: Sandstone images often have complex backgrounds with fillers that can interfere with accurate grain identification.
  • Need for morphological data: Many existing semantic segmentation models don't provide the detailed morphological data (like size and shape) required for mineral analysis.

Methods

The core of this work is an enhanced Mask R-CNN architecture designed for sandstone image segmentation. Key innovations include:

  • Hybrid Attention Mechanism: The model incorporates a combination of Coordinate Attention (CA) and Spatial Attention (SP) mechanisms. CA helps capture long-range spatial dependencies by encoding positional information, while SP focuses on relevant image regions. This hybrid approach aims to improve the model's ability to discern individual grains and their boundaries, especially in complex scenes.
  • Shape-Aware Loss Function: To better fit the contours of irregular sandstone grains, the authors propose using a Log-Cosh DiceLoss. This loss function is designed to be more robust and stable than standard DiceLoss, helping to refine segmentation boundaries and reduce errors, particularly for challenging grain shapes.
  • Dilated Convolution: To increase the receptive field of the convolutional layers without losing resolution, dilated convolutions are employed. This allows the model to capture more global contextual information, which is beneficial for segmenting larger grains and understanding their relationships within the image.
  • Backbone Enhancement: The standard ResNet backbone is augmented with Squeeze-and-Excitation (SE) modules, further improving feature representation by adaptively recalibrating channel-wise feature responses.

Datasets

  • SMISD (Sandstone Microscopic Image Segmentation Dataset): The authors introduce a new, high-quality dataset specifically for sandstone image segmentation. It comprises 288 single-polarized sandstone grain images with over 40,000 labeled grains. To augment this dataset and improve generalization, they applied image enhancement techniques like rotation, resulting in 1121 images with over 153,000 labeled grains.
  • COCO Dataset: The model was also evaluated on the publicly available COCO dataset to demonstrate its general segmentation capabilities and compare it with other state-of-the-art methods on a broader benchmark.

Key Insights and Ideas

  • Attention is Key for Complex Textures: The hybrid attention mechanism (CA+SP) significantly improves the model's ability to distinguish between closely packed or overlapping grains, a common issue in sandstone images.
  • Shape Matters for Loss Functions: Directly optimizing for shape accuracy with a specialized loss function (Log-Cosh DiceLoss) is crucial for achieving fine-grained segmentation of irregular mineral particles.
  • Dilated Convolutions Enhance Context: Expanding the receptive field with dilated convolutions helps the model leverage broader image context, leading to more accurate segmentation, especially for larger grains.
  • Domain-Specific Datasets Drive Performance: The creation and use of the SMISD dataset, tailored to the specific challenges of sandstone analysis, are critical for achieving superior performance compared to models trained solely on general datasets.

Key Results

  • Superior Performance on SMISD: The proposed hybrid attention Mask R-CNN model achieved state-of-the-art results on the SMISD dataset, outperforming conventional Mask R-CNN and other advanced segmentation algorithms. For instance, it showed improvements of 4.3%, 5.2%, and 6.9% in AP, AP50, and AP75 metrics, respectively, compared to the conventional Mask R-CNN.
  • Improved Grain Fitting: Qualitative results show that the improved model provides significantly finer and more accurate segmentation contours, especially for irregular and small grains, compared to the original Mask R-CNN. The model can identify about 80% of grains regardless of size, a substantial improvement over the original's ~10% for small grains.
  • Comparable Performance on COCO: The model demonstrated competitive performance on the general COCO dataset, indicating its robustness and transferability.
  • Effectiveness of Components: Ablation studies confirmed the effectiveness of the hybrid attention mechanism, the shape-aware loss function, and dilated convolution, each contributing to improved segmentation accuracy.

Limitations and Future Work

  • Filler Identification: The paper acknowledges that identifying and segmenting fillers, especially heterogeneous groups and cementation, remains a challenge and an area for future improvement.
  • Recognition Accuracy: While significant progress has been made, the authors suggest there is still room for further improvement in overall recognition accuracy.
  • Data Availability: The paper notes that while the SMISD dataset is valuable, its availability is restricted, implying that broader access could accelerate research in this area.

Summary updated

Abstract

Mineral image segmentation based on computer vision is vital to realize automatic mineral analysis. However, current image segmentation methods still cannot effectively solve the problem of sandstone grains that are adjoined and concealed by leaching processes, and the segmentation performance of small and irregular grains still needs to be improved. This investigation explores and designs a Mask R-CNN-based sandstone image segmentation model, including a hybrid attention mechanism, loss function construction, and receptive field enlargement. Simultaneously, we propose a high-quality sandstone dataset with abundant labels named SMISD to facilitate comprehensive training of the model. The experimental results show that the proposed segmentation model has excellent segmentation performance, effectively solving adhesion and overlap between adjacent grains without affecting the classification accuracy. The model has comparable performance to other models on the COCO dataset, and performs better on SMISD than others.
BibTeX
@inproceedings{dong2024high,
  author = {Lanfang Dong and Hao Gui and Xiaolu Yu and Xinming Zhang and Mingyang Xu},
  title = {{High-Accuracy Image Segmentation Based on Hybrid Attention Mechanism for Sandstone Analysis}},
  booktitle = {Minerals},
  year = {2024},
  doi = {10.3390/min14060544},
}
Figure