Artificial Perceptual Learning: Image Categorization with Weak Supervision

Chengliang Tang, María Uriarte, Helen Jin, Douglas C. Morton, Tian ZhengarXiv 2021 · arXiv

Abstract

Training accurate computer vision models often requires massive, perfectly labeled datasets—a luxury rarely available in specialized fields like ecology. This paper proposes Artificial Perceptual Learning (APL), a bio-inspired framework that mimics infant cognitive development to solve image classification under weak supervision. By combining unsupervised clustering of deep features with sparse, misaligned ground labels, the authors successfully mapped palm tree distributions across a Puerto Rican rainforest with human-level accuracy and high computational efficiency.

Introduction: The Challenge of Weak Supervision

Supervised learning typically demands "pixel-perfect" labels. However, real-world data, particularly in environmental science, often suffers from three types of "weak supervision":

  1. Incomplete: Only a tiny fraction of the data is labeled.
  2. Inexact: Labels are at a different scale (e.g., image-level vs. object-level).
  3. Inaccurate: Labels are noisy or spatially misaligned.

In mapping the El Yunque rainforest, researchers faced all three. Ground truth consisted of trunk locations recorded on the forest floor, which often did not align with tree crowns visible in high-resolution aerial imagery due to tree lean and canopy overlap. Less than 1% of the imagery had any labels at all.

Misalignment between ground-based trunk locations (dots) and aerial canopy segments (polygons).

The APL Framework

The APL framework addresses these challenges by breaking the learning process into four stages designed to emulate how infants develop categories before learning language.

The four-step Artificial Perceptual Learning framework compared to infant categorization.
  1. Visual Feature Engineering: Features are extracted from image patches using a pre-trained VGG-19 network. Custom preprocessing is applied to mitigate noise from shadows, a common issue in tropical imagery.
  2. Unsupervised Learning: Similar to infants grouping objects by perceptual commonalities, APL uses K-means clustering to group similar-looking image patches into "perceptual categories" without initial labels.
  3. Label Assignment: Sparse, noisy ground labels are used to assign a "palm relevance" score to each cluster. By averaging signals across a cluster, the system becomes robust to individual misaligned labels; if a cluster consistently overlaps with ground labels, the entire group is classified as "palm."
  4. Predictive Modeling: The labeled clusters serve as training data for a final, lightweight classifier (XGBoost with HOG features), optimized for speed and deployment.

Datasets and Validation

The study utilized NASA’s G-LiHT high-resolution (3cm) aerial imagery covering 8,000 hectares. Ground truth was provided by the Luquillo Forest Dynamics Plot (LFDP). To validate the model, the authors established a "gold standard" using manual segmentations from Amazon Mechanical Turk workers to compare APL's performance against human effort.

Key Results

The APL framework demonstrated significant advantages in both accuracy and efficiency:

  • Human-Level Accuracy: The model achieved an Intersection over Union (IoU) of 0.49, matching the average performance of human annotators.
  • Operational Speed: The APL classifier processed images in 0.6 seconds, compared to 76 seconds for human manual segmentation—a 125x speedup.
  • Scalability: The system successfully generated a wide-field ecological survey across 8,000 hectares using labels covering less than 1% of the area.

Conclusion

By prioritizing unsupervised feature discovery before label assignment, APL provides a robust solution for fields where high-quality training data is scarce. Future work aims to expand the system to multi-class species identification and incorporate spatial continuity to further refine classification accuracy.

Reference: Tang, C., et al. (2021). Artificial Perceptual Learning: Image Categorization with Weak Supervision. arXiv:2106.07559v1 [cs.CV].

Summary updated

BibTeX
@misc{tang2021artificial,
  author = {Chengliang Tang and María Uriarte and Helen Jin and Douglas C. Morton and Tian Zheng},
  title = {{Artificial Perceptual Learning: Image Categorization with Weak Supervision}},
  year = {2021},
  eprint = {2106.07559},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2106.07559},
}
Figure