Abstract
Weakly Supervised Semantic Segmentation (WSSS) and Object Localization (WSOL) typically rely on Class Activation Mapping (CAM) to generate pseudo-labels from image-level annotations. However, standard CAM methods are sensitive to the stability of training outcomes, such as gradients and prediction scores, which become highly unreliable in small-scale data regimes. We introduce BroadCAM, an outcome-agnostic framework that decouples feature importance from the deep model's internal classification logic. By utilizing an independent Broad Learning System (BLS) to weight multi-layer features, BroadCAM achieves superior localization performance in data-scarce environments. Experiments on VOC2012, BCSS-WSSS, and OpenImages30k demonstrate that BroadCAM significantly outperforms traditional CAM methods in data-scarce scenarios while maintaining competitive performance on large-scale datasets.
Introduction: The Small-Data Bottleneck
The effectiveness of CAM-based methods is often predicated on the availability of large-scale datasets. In domains like medical imaging or niche industrial inspection, data is often scarce. In these scenarios, deep models suffer from unstable training, leading to noisy gradients. Since traditional CAM methods (e.g., Grad-CAM) rely on these gradients to weight feature maps, the resulting object "seeds" are often inaccurate or "messy."
BroadCAM addresses this by treating the deep model solely as a feature extractor and employing an external, robust classifier to determine feature relevance. This "outcome-agnostic" approach ensures that localization quality is not tied to the convergence stability of the primary deep model.
Methodology
BroadCAM's architecture consists of three primary stages: multi-layer feature extraction, broad learning weight calculation, and seed generation.
1. Multi-layer Feature Extraction
Unlike standard CAM which typically uses only the final convolutional layer, BroadCAM extracts feature maps from multiple stages of a CNN backbone (e.g., ResNet). This allows the model to capture both coarse semantic information and fine-grained textural details, which are essential for precise object boundaries.
2. Broad Learning System (BLS) Weighting
The core innovation is the use of a Broad Learning System (BLS) to generate localization weights. The BLS is a flat network that maps features into "feature nodes" and "enhancement nodes." It is trained via a one-round ridge regression, making it significantly more stable and efficient than backpropagation-based deep heads when samples are limited. This ensures that the weights reflect the true relevance of feature maps to the target category, independent of the deep model's classification performance.
3. Seed Generation
The final CAM is produced by a weighted sum of the multi-layer feature maps. While weights are calculated for all layers, empirical results suggest that aggregating the final two layers (L3 and L4) provides the optimal balance between object coverage and boundary precision.
Key Results and Discussion
BroadCAM was evaluated across natural (VOC2012), medical (BCSS-WSSS), and general object (OpenImages30k) datasets to prove its versatility.
- Small-Scale Dominance: On VOC2012 with only 1% of training data, BroadCAM achieved a Mean Intersection over Union (mIoU) of 46.55%, a significant improvement over the 33.85% achieved by traditional CAM.
- Medical Accuracy: In the BCSS histopathology dataset, BroadCAM outperformed competing methods by nearly 10% in Frequency Weighted IoU (FwIoU) at the 1% data level, highlighting its potential for clinical applications where annotated data is difficult to obtain.
- Consistency: Unlike other methods that see a massive performance drop as data scales down, BroadCAM’s performance degrades gracefully. Furthermore, it remains competitive with state-of-the-art methods even when 100% of the data is available.
Conclusion
BroadCAM provides a robust solution for weakly supervised learning in data-constrained environments. By decoupling localization from unstable training outcomes through the use of a Broad Learning System, it enables the generation of high-quality object seeds where traditional methods fail. Future work will focus on more sophisticated ways to denoise shallow feature maps to capture even finer object boundaries without losing semantic focus.
Summary updated