Testing the Channels of Convolutional Neural Networks

Kang Choi, Donghyun Son, Younghoon Kim, Jiwon Seo

Abstract

This paper presents a novel "unit testing" framework for Convolutional Neural Networks (CNNs) by conceptualizing internal channels as modular units. We introduce FtGAN (Feature-testing GAN), a generative model capable of producing realistic, semantic test cases by precisely manipulating specific channel intensities. Our approach enables the systematic identification of hidden semantic defects—such as a face recognizer failing specifically on individuals with green hair—which traditional adversarial testing often misses. By treating channels as logical modules, we move beyond low-level neuron coverage toward meaningful semantic debugging.

Introduction: The Black Box Problem

Deep learning models are notoriously difficult to debug due to their "black box" nature. In traditional software engineering, modularity allows developers to test individual functions or components. In contrast, CNNs are typically tested as monolithic entities. While adversarial attacks can find pixel-level perturbations that fool a model, these rarely correspond to real-world concepts. Furthermore, existing metrics like "neuron coverage" are often too low-level to provide actionable insights for developers.

The authors argue that channels (feature maps) are the true logical units of a CNN, representing higher-level concepts such as textures, shapes, and colors. The core challenge addressed here is how to systematically manipulate these internal channels to observe where a model’s logic breaks down.

Methodology

The proposed framework for channel-wise testing consists of three primary components: FtGAN for image generation, a greedy selection algorithm for efficiency, and an unexpectedness score for automated bug detection.

FtGAN (Feature-testing GAN)

To "unit test" a specific channel, the authors developed FtGAN. This modified GAN is designed to "tame" the latent space of a target CNN. As shown in the architecture below, FtGAN takes an input image $x$ and a target intensity $I$ for a specific channel. It uses an encoder-decoder structure ($G_{enc}$ and $G_{dec}$) to generate a new version of the image, $x'$, where the intensity of the target channel is controlled while preserving the original image's identity.

Overview of the FtGAN architecture, illustrating the use of reconstruction, adversarial, and channel intensity losses to control specific feature maps.

The training process involves a combination of:

  1. Reconstruction Loss: Ensures $x'$ remains visually similar to $x$.
  2. Adversarial Loss: Ensures the generated image $x'$ is realistic.
  3. Channel Intensity Loss: Forces the target channel in the victim network to match the desired intensity $I$.

Greedy Channel Selection

Modern CNNs contain thousands of channels, making exhaustive testing computationally prohibitive. The authors treat channel selection as a "minimal hitting set" problem. They employ a greedy algorithm to identify a small subset of representative channels that are highly correlated with others, ensuring high coverage of the network's internal states with minimal test cases.

Unexpectedness Score

To automate the detection of potential bugs, the framework calculates an "unexpectedness" score. This metric measures the $L1$ distance between the channel correlations observed in the test data versus those learned during training. If a test image causes internal channels to fire in a way that contradicts established training patterns, it is flagged as a potential defect.

Experimental Evaluation

The researchers evaluated their approach across diverse domains, including digit recognition (MNIST, SVHN), face identification (VGG Face, ResNet-50), and autonomous driving (CARLA simulator).

Bug Discovery and Semantic Variations

FtGAN successfully identified "data corruption bugs." In one experiment, a model was intentionally poisoned to misidentify individuals with green hair. FtGAN's channel-wise testing quickly flagged the specific channel responsible for this error by generating variations of the same face with different hair colors.

Semantic variations generated by FtGAN for MNIST digits and CelebA faces, highlighting the ability to isolate specific features like "green hair" for testing.

In the CARLA autonomous driving environment, the system revealed that steering decisions were dangerously sensitive to specific road textures and lighting changes—vulnerabilities that standard noise-based testing tools like TensorFuzz failed to uncover.

Key Insights

  • Semantic Debugging: Unlike standard adversarial noise, FtGAN produces variations that humans can interpret. This allows developers to understand why a model fails (e.g., "The model fails when road lines are worn") rather than just knowing that it fails.
  • Inconsistency as a Signal: The paper leverages the idea that neural networks learn implicit invariants. When FtGAN creates a scenario that breaks these invariants, the resulting "unexpectedness" serves as a strong indicator of a model's blind spot.
  • Efficiency: The greedy selection algorithm achieved significantly higher "channel boundary coverage" compared to random selection or standard adversarial methods like IGSM. Furthermore, fine-tuning FtGAN for a specific channel is relatively fast, taking between 10 minutes and an hour depending on the dataset complexity.

Limitations and Future Work

The method's primary limitation is its dependence on the underlying GAN's capabilities; if a GAN has not encountered a specific attribute during its own training, it cannot generate test cases for it. Additionally, while the unexpectedness score helps rank results, a human is still required to verify if the flagged behavior is a genuine bug. Future research may explore more automated ways to test complex interactions between multiple channels simultaneously, moving beyond the current "chaining" approach.

Summary updated

BibTeX
@inproceedings{choi2023testing,
  author = {Kang Choi and Donghyun Son and Younghoon Kim and Jiwon Seo},
  title = {{Testing the Channels of Convolutional Neural Networks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i12.26726},
}
Figure