Automatic Image Labelling at Pixel Level

Xiang Zhang, Wei Zhang, Jinye Peng, Jianping FanarXiv 2020 · arXiv

Abstract

The Guided Filter Network (GFN) is a novel framework designed to automatically generate high-quality, pixel-level image labels, drastically reducing the need for manual annotation in semantic segmentation. By transferring segmentation knowledge from source domains and using an iterative "self-improvement" loop, GFN produces fine-grained masks that rival human-labeled ground truth.

Introduction

Deep Convolutional Neural Networks (DCNNs) have revolutionized semantic segmentation, but they are data-hungry. To perform well, they require massive datasets with precise, pixel-level labels. Creating these labels is a "dreadfully labor-intensive" process that often results in human inconsistency. While "weakly-supervised" methods exist—using only bounding boxes, scribbles, or image-level tags—they often struggle to capture precise object boundaries. The Guided Filter Network (GFN) addresses this bottleneck by generating accurate "pseudo-labels" to train high-performance segmentation models without manual intervention.

Methodology

The GFN operates in two main stages: a proxy stage (learning from a source) and an iterative learning stage (refining in the target). The architecture integrates three key modules to ensure high-quality mask generation.

The GFN architecture integrates the Boundary Guided (BG), High-level Guided Low-level (HGL), and Guided Filter (GF) modules to refine segmentation masks.

Core Modules

  1. Boundary Guided (BG) Module: This module highlights object edges using a boundary extractor. It calculates the difference between max-pooling and min-pooling operations to identify sharp transitions in feature maps.
  2. High-level Guided Low-level (HGL) Module: To mitigate noise in low-level features, this module uses high-level semantic features as a gate, ensuring the network focuses on relevant structural details.
  3. Guided Filter (GF) Module: An edge-preserving component that uses the original image as a guide to sharpen the final mask, aligning it perfectly with object boundaries.
The BG module uses a simple yet effective boundary extractor based on the difference between max-pooling and min-pooling.

Training Strategies

To initialize the system without target labels, GFN employs three pretext tasks:

  • Transfer Learning: Leveraging knowledge from semantically similar source domains.
  • Simple-to-Complex: Training on images with simple backgrounds (using Otsu’s method) before tackling complex scenes.
  • Class Activation Maps (CAMs): Refining image-level labels into initial regional masks.

The process concludes with Iterative Learning, where coarse masks are treated as pseudo-labels to progressively re-train and refine the network.

Results

GFN was evaluated across six diverse datasets, including PASCAL VOC 2012, FGVC Aircraft, and CUB-200-2011.

  • Quantitative Performance: On PASCAL VOC 2012, GFN achieved a mean Intersection over Union (mIoU) of 62.3%, surpassing many state-of-the-art weakly-supervised models. On specialized datasets like FGVC Aircraft, it reached an mIoU of 91.5%.
  • Qualitative Quality: Visual results show that the predicted masks are remarkably sharp, capturing intricate structures like bird beaks and tree branches that typically challenge automated systems.
  • Scalability: Performance improved significantly as more unlabelled "simple" images were added, demonstrating the framework's ability to scale with data.

Conclusion

The Guided Filter Network provides a robust solution for automatic pixel-level labeling. By combining boundary guidance with semantic filtering and iterative self-correction, it bridges the gap between weakly-supervised methods and manual annotation. Future work may explore integration with Transformer-based backbones and more sophisticated reliability metrics to further mitigate semantic drift during iterative learning.

Summary updated

Abstract

The performance of deep networks for semantic image segmentation largely depends on the availability of large-scale training images which are labelled at the pixel level. Typically, such pixel-level image labellings are obtained manually by a labour-intensive process. To alleviate the burden of manual image labelling, we propose an interesting learning approach to generate pixel-level image labellings automatically. A Guided Filter Network (GFN) is first developed to learn the segmentation knowledge from a source domain, and such GFN then transfers such segmentation knowledge to generate coarse object masks in the target domain. Such coarse object masks are treated as pseudo labels and they are further integrated to optimize/refine the GFN iteratively in the target domain. Our experiments on six image sets have demonstrated that our proposed approach can generate fine-grained object masks (i.e., pixel-level object labellings), whose quality is very comparable to the manually-labelled ones. Our proposed approach can also achieve better performance on semantic image segmentation than most existing weakly-supervised approaches.
BibTeX
@misc{zhang2020automatic,
  author = {Xiang Zhang and Wei Zhang and Jinye Peng and Jianping Fan},
  title = {{Automatic Image Labelling at Pixel Level}},
  year = {2020},
  eprint = {2007.07415},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2007.07415},
}
Figure