Testing the Channels of Convolutional Neural Networks

Kang Choi, Donghyun Son, Younghoon Kim, Jiwon Seo

Abstract

Deep learning models often operate as "black boxes," making them difficult to debug and verify using traditional software engineering principles. This paper proposes a novel framework that treats internal convolutional neural network (CNN) channels as logical units for "unit testing." By leveraging a specialized generative model, FtGAN, to manipulate channel intensities and an "unexpectedness score" to detect internal logic shifts, the authors identify semantic defects and data-poisoning vulnerabilities. Experimental results across diverse datasets, including autonomous driving simulators and facial recognition tasks, demonstrate that this channel-centric approach uncovers bugs that traditional adversarial testing and neuron-level fuzzing fail to detect.

1. Introduction

The verification of Deep Neural Networks (DNNs) remains a significant hurdle in the deployment of safety-critical AI systems. In traditional software engineering, "unit testing" allows developers to isolate and verify individual modules. However, neural networks lack clear modular boundaries. Previous attempts at testing have focused either on low-level neuron activations—which are often too granular to carry semantic meaning—or on end-to-end adversarial attacks that treat the model as a single monolithic entity.

This research introduces a middle-ground approach: treating channels (feature maps) as the fundamental logical units of a CNN. Because channels often correspond to specific semantic concepts (e.g., textures, shapes, or colors), they provide a more interpretable level of abstraction for debugging. The proposed framework enables systematic testing of these units to identify "semantic bugs" and biases introduced during training.

2. Methodology

The framework consists of three core components designed to isolate, manipulate, and monitor the behavior of CNN channels.

2.1 FtGAN: Feature-Testing Generative Adversarial Network

To test a specific channel, one must be able to generate inputs that trigger that channel in a controlled manner. The authors developed FtGAN, an extension of the GAN architecture. FtGAN is trained to generate test images where the intensity of a target channel in the "model under test" can be precisely adjusted via a scalar input $I$. By "turning the knob" on $I$, researchers can observe how the model's final output changes as a specific internal feature is amplified or suppressed.

2.2 Greedy Channel Selection

Modern CNNs contain thousands of channels, making exhaustive testing computationally prohibitive. To address this, the authors frame channel selection as a "minimal hitting set" problem. They utilize Pearson correlation to identify a subset of "representative" channels. These channels are highly correlated with the broader population, allowing for high coverage of the model's internal state space with a significantly reduced number of tests.

2.3 The Unexpectedness Score

Automated bug detection requires a metric to identify when a model is behaving incorrectly. The authors propose the Unexpectedness Score, which measures the deviation of internal channel relationships. During training, certain channels tend to fire together in consistent patterns. If a test input causes these pair-wise correlations to shift significantly from the learned distribution, the model is likely in an "unexpected" state, signaling a potential defect or vulnerability to out-of-distribution data.

3. Experimental Evaluation

The framework was evaluated on five datasets: MNIST, SVHN, VGG Face, CelebA, and the CARLA autonomous driving simulator. Additionally, a modified "Morpho-MNIST" was used to simulate synthetic data-poisoning defects.

3.1 Semantic Bug Discovery

Unlike standard adversarial attacks that add imperceptible pixel noise, FtGAN generates semantic variations. For example, in facial recognition tasks, FtGAN could isolate a channel representing "green hair." If the model consistently misclassifies individuals with this attribute, the framework flags it as a semantic bug. This allows developers to understand why a model fails (e.g., a bias against specific hair colors) rather than simply knowing that it fails.

3.2 Autonomous Driving and Safety

In the CARLA simulator, the tool was used to analyze steering decisions. The framework identified that specific channels related to "road texture" and "center line wear-out" were critical failure points. When these channels were manipulated, the model made incorrect steering decisions, revealing a lack of robustness to common road degradation—a finding that end-to-end testing might have missed.

4. Results and Discussion

The "unit testing" approach demonstrated several advantages over existing baselines:

  • Coverage: Compared to TensorFuzz, a popular coverage-guided fuzzer, the proposed method achieved significantly higher feature coverage. TensorFuzz often covered less than 35% of the features in the tested CNNs, whereas the greedy selection method targeted the most influential logical units.
  • Data Poisoning Detection: The framework successfully identified "data corruption bugs" where models were trained on biased or poisoned data.
  • Efficiency: The greedy selection algorithm made the process computationally feasible, proving that one does not need to test every neuron to ensure model reliability.

5. Limitations and Future Work

While effective, the framework is dependent on the capabilities of the underlying GAN. FtGAN can only manipulate attributes present within the training distribution; it cannot simulate entirely novel types of corruption. Furthermore, while the unexpectedness score ranks potential bugs, a human expert is still required to verify if the flagged behavior constitutes a true defect. Future research may explore multi-channel interactions and the "chaining" of tests to simulate more complex failure modes.

6. Conclusion

By treating CNN channels as modular units, this research bridges the gap between deep learning and traditional software testing. The combination of FtGAN-driven semantic manipulation and correlation-based unexpectedness scoring provides a powerful toolkit for debugging the "black box" of neural networks, moving us closer to verifiable and robust AI systems.

Summary updated

BibTeX
@inproceedings{choi2023testing,
  author = {Kang Choi and Donghyun Son and Younghoon Kim and Jiwon Seo},
  title = {{Testing the Channels of Convolutional Neural Networks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i12.26726},
}
Figure