Introspection of 2D Object Detection using Processed Neural Activation Patterns in Automated Driving Systems

Hakan Yekta Yatbaz, Mehrdad Dianati, Konstantinos Koufos, Roger WoodmanWorkshops @ ICCV 2023 · DOI

Abstract

Deep Neural Networks (DNNs) serve as the primary perception modules in modern Automated Driving Systems (ADS). However, their performance can degrade under adverse environmental conditions, posing significant safety risks. This paper investigates a framework for "introspection"—the ability of a system to monitor its own internal state and identify potential perception errors. By applying Activation Shaping (ASH) techniques to prune noisy neural activation patterns from an object detector's backbone, we demonstrate that the "signatures" of failure become more discernible. Our results on the Berkeley Deep Drive (BDD) dataset show a 14% reduction in the False Negative Rate of error detection compared to using raw activations, outperforming current state-of-the-art methods.

Introduction

The reliability of Automated Driving Systems (ADS) hinges on the accuracy of their perception systems. While Deep Neural Networks (DNNs) have achieved remarkable success in object detection, they remain vulnerable to "corner cases" such as heavy precipitation, extreme lighting, or rare traffic scenarios. To mitigate the risks associated with these failures, introspection mechanisms are required. Introspection allows the ADS to detect when its primary perception model is likely to fail, enabling a safe transition of control to a human driver (Level 3 autonomy) or the execution of a minimum risk maneuver (Level 4 autonomy).

Traditional introspection methods often rely on external metrics or raw internal signals. However, raw neural activation patterns—the internal feature maps generated by the network—are often high-dimensional and contain significant noise, making it difficult for a secondary monitoring network to distinguish between successful and erroneous states. This research proposes a method to "shape" these activations to enhance the reliability of error detection.

Methodology

The proposed framework consists of a primary object detector and a secondary introspection module. The object detector used in this study is the Fully Convolutional One-Stage (FCOS) detector. The introspection module is a ResNet18 architecture trained to classify the detector's performance as either "success" (mAP ≥ 0.5) or "failure" (mAP < 0.5).

The proposed framework linking the object detection pipeline with the introspection module via processed neural activation patterns.

The core innovation lies in the Activation Pre-processing stage, where Activation Shaping (ASH) is applied to the feature maps extracted from the final layer of the detector's backbone. Three variants of ASH were evaluated:

  • ASH-P (Pruning): Activations below a certain percentile (e.g., 75th or 90th) are zeroed out, removing low-energy noise.
  • ASH-B (Binarization): Following pruning, all remaining non-zero activations are set to a constant value, focusing purely on the spatial pattern of high-energy signals.
  • ASH-S (Scaling): Pruned activations are scaled to ensure the total energy of the feature map remains consistent with the original.

Experimental Setup

The framework was evaluated using two benchmark datasets:

  • KITTI: A dataset of urban driving in Germany (~14,000 images). Due to its smaller size and high class imbalance, it served as a preliminary testbed.
  • Berkeley Deep Drive (BDD): A large-scale, diverse dataset (100,000 images) featuring varied weather, lighting, and geographic locations. This dataset provided a robust environment for testing the generalizability of the introspection model.

For consistency across datasets, object labels were grouped into "vehicle" and "people" categories.

Results and Discussion

The experimental results highlight the effectiveness of pruning in clarifying internal error signals. On the BDD dataset, the use of ASH-P at the 75th percentile reduced the False Negative Rate (FNR)—the rate at which the system fails to identify a perception error—by 14% compared to using raw, unprocessed activations.

Furthermore, the ASH-based introspection method achieved a 3% improvement in overall detection accuracy (measured by AUROC) over existing state-of-the-art (SOTA) methods that utilize basic statistical features. While ASH-S (scaling) showed promise in some configurations, ASH-P (pruning) emerged as the most stable and effective technique across different thresholds. This suggests that "less is more" when it comes to neural introspection; by removing the majority of internal signals and focusing on the most prominent activations, the introspection network can more accurately identify the patterns associated with perception failure.

Conclusion

This study demonstrates that processing internal neural signals via Activation Shaping significantly enhances the ability of Automated Driving Systems to detect their own perception errors. Despite these gains, challenges remain regarding computational latency and domain shift robustness. Future work will explore the sensitivity of pruning thresholds across different detector architectures and the performance of the introspector under extreme environmental transitions.

Summary updated

BibTeX
@inproceedings{yatbaz2023introspection,
  author = {Hakan Yekta Yatbaz and Mehrdad Dianati and Konstantinos Koufos and Roger Woodman},
  title = {{Introspection of 2D Object Detection using Processed Neural Activation Patterns in Automated Driving Systems}},
  booktitle = {Workshops @ ICCV},
  year = {2023},
  doi = {10.1109/iccvw60793.2023.00437},
}
Figure