Deep Inspection: An Electrical Distribution Pole Parts Study VIA Deep Neural Networks

Liangchen Liu, Teng Zhang, Kun Zhao, Arnold Wiliem, Kieren Astin-Walmsley, Brian C. Lovell

TL;DR

This paper introduces a novel deep learning framework for automating the inspection of electrical distribution poles, specifically addressing the challenges of detecting tiny components and handling highly imbalanced datasets. Their proposed two-stage "zoom-in" detection method and data balancing techniques significantly improve the accuracy of identifying critical pole parts like pole caps.

What Problem Does This Paper Address?

Maintaining the vast network of electrical distribution poles is crucial for public safety and reliable power supply. Traditional inspection methods, relying on manual analysis of aerial imagery (from helicopters or drones), are time-consuming, labor-intensive, and expensive. While automated inspection systems using deep learning show promise, two major hurdles hinder their widespread adoption:

  1. Tiny Object Detection: Many critical components on a distribution pole, such as the pole cap, are very small in the captured images, making them difficult for standard object detection algorithms to identify accurately.
  2. Extremely Imbalanced Datasets: In a typical inspection dataset, the vast majority of components are in a normal state (e.g., pole cap present), while abnormal conditions (e.g., pole cap missing) are rare. This severe imbalance makes it challenging for deep learning models to learn to detect these infrequent but important anomalies.

Methods

The authors propose an integrated framework to tackle these challenges, focusing on two key innovations:

  1. Two-Stage Zoom-In Detection: To address the tiny object detection problem, they introduce a sequential approach. First, a coarse detector identifies the entire distribution pole. Then, a second, more focused detector is applied to a cropped sub-region of the pole (specifically the upper half where the pole cap is located). This "zooming in" effectively increases the relative size of the target object (the pole cap) within the input to the second detector, making it easier to find. They leverage the Faster R-CNN architecture for both stages.
  2. Negative Sample Resampling and Reweighting: To combat the imbalanced dataset issue, they employ two complementary techniques:
  3. Resampling: They iteratively train the model by balancing the number of negative samples with the number of positive samples. This involves selecting a subset of negative samples that matches the size of the positive set for each training iteration.
  4. Reweighting: This technique modifies the loss function by assigning a higher weight to misclassifications of minority class samples (e.g., missing pole caps). This forces the model to pay more attention to these rare but critical cases.

Datasets

The paper utilizes a dataset of electrical distribution pole images. While the total dataset size is 103,649 entries, the authors manually labeled 428 image samples for specific objects like pole caps, cross arms, and poles. These labeled samples were split into a training set (216 images) and a testing set (212 images) for the pole cap detection experiments. For classification tasks, they also used statistics from their sample dataset to identify imbalanced classes, such as pole types (e.g., Aluminum, Concrete, Wood).

Key Insights and Ideas

  • Hierarchical Detection is Key for Small Objects: The "zoom-in" approach demonstrates that breaking down the detection task into stages—first finding the general object (the pole) and then focusing on a specific part of it—is highly effective for small object detection. This is more efficient than trying to find tiny objects in a massive, high-resolution image directly.
  • Data Imbalance is a Major Bottleneck: The paper strongly emphasizes that standard deep learning models struggle significantly with datasets where positive examples are orders of magnitude rarer than negative ones. This is a common problem in anomaly detection and specialized inspection tasks.
  • Iterative Balancing Works: Both resampling and reweighting, when applied iteratively, show substantial improvements in performance metrics like AUC, suggesting that models can adapt to imbalanced data with repeated exposure and adjusted learning signals.
  • Transfer Learning is a Necessary Starting Point: The authors leverage pre-trained VGG16 models (from ImageNet and COCO datasets) as a foundation, highlighting that starting with a model that already understands general image features is crucial, especially when dealing with limited labeled data for specific tasks.

Key Results

  • Two-Stage Zoom-In Detection: Achieved a Mean Average Precision (mAP) of 0.796, significantly outperforming the baseline detection method (mAP of 0.507) for pole cap detection.
  • Resampling Approach: For pole cap missing classification, the AUC score improved significantly with iterative resampling, reaching 0.9 after 5 loops, compared to an initial AUC of 0.63. Similar improvements were observed for wood pole type classification.
  • Reweighting Approach: The reweighting method also consistently improved performance over the baseline, demonstrating its efficacy in handling imbalanced data. For instance, with a 1:12 imbalance ratio, the AUC improved from 0.853 (baseline) to 0.992 (reweighting).

Limitations and Future Work

  • Manual Labeling Effort: The paper acknowledges that manual labeling of objects was required for their experiments, which can be a bottleneck for creating large-scale datasets.
  • Intra-Class Variation: For the wood pole type classification, the resampling approach showed improvement but not as dramatic as for pole caps. The authors suspect this is due to large intra-class variations within the negative samples and potentially insufficient resampling iterations.
  • Future Directions: The authors plan to:
  • Jointly fuse the resampling and reweighting approaches for even better performance.
  • Explore multi-modal data (e.g., combining visual data with other sensor information) to enhance the framework.
  • Develop automatic keyword or report generation functionalities for the inspection system.

Summary updated

Abstract

Electrical distribution poles are important assets in electricity supply. These poles need to be maintained in good condition to ensure they protect community safety, maintain reliability of supply, and meet legislative obligations. However, maintaining such a large volumes of assets is an expensive and challenging task. To address this, recent approaches utilise imagery data captured from helicopter and/or drone inspections. Whilst reducing the cost for manual inspection, manual analysis on each image is still required. As such, several image-based automated inspection systems have been proposed. In this paper, we target two major challenges: tiny object detection and extremely imbalanced datasets, which currently hinder the wide deployment of the automatic inspection. We propose a novel two-stage zoom-in detection method to gradually focus on the object of interest. To address the imbalanced dataset problem, we propose the resampling as well as reweighting schemes to iteratively adapt the model to the large intra-class variation of major class and balance the contributions to the loss from each class. Finally, we integrate these components together and devise a novel automatic inspection framework. Extensive experiments demonstrate that our proposed approaches are effective and can boost the performance compared to the baseline methods.
BibTeX
@inproceedings{liu2019deep,
  author = {Liangchen Liu and Teng Zhang and Kun Zhao and Arnold Wiliem and Kieren Astin-Walmsley and Brian C. Lovell},
  title = {{Deep Inspection: An Electrical Distribution Pole Parts Study VIA Deep Neural Networks}},
  booktitle = {ICIP},
  year = {2019},
  url = {https://openreview.net/forum?id=R7YNYA-jVS5j},
}
Figure