Joint Attention-Guided Feature Fusion Network for Saliency Detection of Surface Defects

Xiaoheng Jiang, Feng Yan, Yang Lu, Ke Wang, Shuai Guo, Tianzhu Zhang, Yanwei Pang, Jianwei Niu, Mingliang XuIEEE Trans. Instrumentation & Measurement 2022 · DOI · OpenReview

TL;DR

This paper introduces JAFFNet, a novel deep learning architecture for detecting surface defects in industrial settings. It tackles challenges like varying defect scales and low contrast by employing a joint attention mechanism to fuse features and a dense receptive field module for richer context, achieving state-of-the-art performance and real-time detection.

What Problem Does This Paper Address?

Surface defect inspection is crucial for industrial quality control. While deep learning has advanced this field, existing methods struggle with several real-world challenges:

  • Scale Variation: Defects can range from tiny imperfections to large blemishes, making it hard for models to capture them all.
  • Low Contrast: Defects can have very subtle differences in color or texture compared to the background, making them difficult to distinguish, especially under varying lighting conditions.
  • Background Interference: Non-defect features in the background can sometimes resemble defects, leading to false positives.

Traditional methods often rely on hand-crafted features and struggle with the complexity and diversity of defects. While CNN-based approaches have improved, simply combining low-level (detail-rich) and high-level (semantic) features can lead to defect features being drowned out by background noise, especially for subtle defects.

Methods

The proposed JAFFNet is an encoder-decoder network enhanced with two novel modules:

  • Joint Attention-Guided Feature Fusion (JAFF) Module: This module is integrated into the decoder stages to adaptively fuse low-level and high-level features. It uses a dual attention mechanism (combining channel and spatial attention) derived from high-level features to generate an attention map. This map then guides the fusion process by emphasizing relevant defect features and suppressing background noise. This is a key innovation, moving beyond simple concatenation or addition of features.
  • Dense Receptive Field (DRF) Module: Placed after the encoder, this module is designed to capture context information across a wide range of scales. It uses a cascade of multi-receptive-field (MRF) units that are densely connected. This allows the module to integrate information from various receptive fields, effectively handling defects of different sizes.

The overall architecture uses a modified ResNet18 as the encoder, extracting multi-level features. The decoder then progressively fuses these features using the JAFF module.

Datasets

The paper evaluates JAFFNet on three publicly available surface defect datasets:

  • SD-saliency-900: Contains typical surface defects like patch, inclusion, and scratch, known for scale variation, low contrast, and complex backgrounds.
  • Magnetic tile: Features various defect types (uneven, fray, crack, blowhole, break) with complex backgrounds and lighting variations.
  • DAGM 2007: Includes 10 defect types with challenges like low contrast and small defect sizes.

Key Insights and Ideas

  • Attention for Feature Fusion: The core idea is that high-level semantic features can "teach" low-level features where to focus, effectively highlighting subtle defects and ignoring irrelevant background details. This is more sophisticated than simple feature concatenation.
  • Dense Receptive Fields for Scale: The DRF module's dense connectivity of MRF units is crucial for capturing context at multiple scales, a common challenge in defect detection.
  • End-to-End Saliency Detection: The network is designed for end-to-end saliency detection, directly outputting a map indicating defect likelihood, which is highly desirable for automated inspection.

Key Results

  • State-of-the-Art Performance: JAFFNet achieves superior performance across the three benchmark datasets (SD-saliency-900, Magnetic tile, and DAGM 2007) compared to several other state-of-the-art methods, as evidenced by higher F-measure, Sm, and Em scores, and lower MAE. For instance, on SD-saliency-900, it shows significant improvements over existing methods.
  • Real-time Detection: The model achieves a fast detection speed of 66 FPS on a single RTX 3060 Ti GPU, making it suitable for real-time industrial applications.
  • Ablation Studies: Experiments confirm the effectiveness of both the JAFF module (especially the dual attention mechanism) and the DRF module. Replacing them with simpler alternatives or other attention mechanisms resulted in performance degradation.

Limitations and Future Work

  • Failure Cases: The paper acknowledges that the model can still struggle with extremely challenging defects, sometimes misclassifying background regions as defects or missing very tiny defects. This is partly attributed to the scarcity of training samples for certain defect types, affecting generalization.
  • Model Complexity: While achieving high performance, the model has more parameters and computational cost compared to some lighter-weight methods.
  • Future Directions: The authors aim to reduce the model's complexity through network compression techniques to make it more lightweight and practical for resource-constrained industrial environments. They also suggest that addressing the scarcity of defect samples for certain types could further improve generalization.

Summary updated

Abstract

Surface defect inspection plays an important role in the process of industrial manufacture and production. Though convolutional neural network (CNN)-based defect inspection methods have made huge leaps, they still confront a lot of challenges such as defect scale variation, complex background, low contrast, and so on. To address these issues, we propose a joint attention-guided feature fusion network (JAFFNet) for saliency detection of surface defects based on the encoder–decoder network. JAFFNet mainly incorporates a joint attention-guided feature fusion (JAFF) module into decoding stages to adaptively fuse low-level and high-level features. The JAFF module learns to emphasize defect features and suppress background noise during feature fusion, which is beneficial for detecting low-contrast defects. In addition, JAFFNet introduces a dense receptive field (DRF) module following the encoder to capture features with rich context information, which helps detect defects of different scales. The JAFF module mainly utilizes a learned joint channel-spatial attention map provided by high-level semantic features to guide feature fusion. The attention map makes the model pay more attention to defect features. The DRF module utilizes a sequence of multireceptive-field (MRF) units with each taking as inputs all the preceding MRF feature maps and the original input. The obtained DRF features capture rich context information with a large range of receptive fields. Extensive experiments conducted on SD-saliency-900, magnetic tile, and German Association for Pattern Recognition (DAGM) 2007 indicate that our method achieves promising performance in comparison with other state-of-the-art methods. Meanwhile, our method reaches a real-time defect detection speed of 66 FPS.
BibTeX
@article{jiang2022joint,
  author = {Xiaoheng Jiang and Feng Yan and Yang Lu and Ke Wang and Shuai Guo and Tianzhu Zhang and Yanwei Pang and Jianwei Niu and Mingliang Xu},
  title = {{Joint Attention-Guided Feature Fusion Network for Saliency Detection of Surface Defects}},
  journal = {IEEE Trans. Instrumentation \& Measurement},
  year = {2022},
  doi = {10.1109/tim.2022.3218547},
}
Figure