Dynamic slicing for deep neural networks

Ziqi Zhang, Yuanchun Li, Yao Guo, Xiangqun Chen, Yunxin Liu

TL;DR

This paper introduces NNSlicer, the first dynamic program slicing technique designed specifically for Deep Neural Networks (DNNs). By tracking data flow and comparing neuron activations against their historical averages, NNSlicer identifies the critical sub-network (neurons and synapses) responsible for a specific prediction. The authors demonstrate its utility across three major tasks: detecting adversarial inputs, targeted model pruning, and selective model protection.

What Problem Does This Paper Address?

In traditional software engineering, "program slicing" is a classic debugging technique used to isolate the exact lines of code that affect a specific variable. However, applying this concept to Deep Neural Networks (DNNs) is incredibly difficult. Unlike traditional programs with human-readable instructions and variables, DNNs are dense, "black-box" graphs of meaningless mathematical operations and learned weights.

Because every output in a DNN is technically connected to almost every neuron in the preceding layers, standard data-flow analysis fails—everything looks important. Yet, we desperately need a way to slice DNNs to interpret their decisions, prune their massive sizes for edge deployment, and protect valuable model IP without encrypting the entire network. This paper asks: How can we meaningfully extract the exact subset of neurons and synapses that drive a specific DNN output?

Methods

To solve the dense-graph problem, the authors propose NNSlicer, a tool that computes dynamic slices (input-dependent sub-networks) using a three-phase approach:

  1. Profiling Phase (The Baseline): NNSlicer feeds the entire training dataset through the model to calculate the average activation value for every single neuron. This establishes a "behavioral standard" for what a neuron normally does.
  2. Forward Analysis Phase (The Reaction): When a target input is fed into the network, NNSlicer records the activation values and subtracts the baseline averages computed in step 1. This difference (relative activation) reveals how sensitively a neuron reacts to this specific input.
  3. Backward Analysis Phase (The Contribution): Starting from the output neurons of interest (the "slicing criterion"), NNSlicer backtracks through the network. It calculates the "contribution" of preceding neurons by multiplying the target neuron's relative activation, the connecting synapse's weight, and the preceding neuron's relative activation. Synapses and neurons with the highest cumulative contributions are kept; the rest are ignored.

Datasets

The paper primarily evaluates NNSlicer using the CIFAR-10 image classification dataset (32x32 images). For the adversarial defense evaluation, the authors generated adversarial examples using 17 different state-of-the-art attack methods (including FGSM, DeepFool, PGD, CW, and Boundary attacks) via the Foolbox library.

Key Insights and Ideas

  • Relative Activation is the Secret Sauce: Just looking at raw activation values isn't enough to understand a neuron's importance, because some neurons naturally output larger numbers. By comparing an activation to its dataset average, NNSlicer performs a sort of "differential power analysis" for neural networks, isolating the true signal from the noise.
  • Slices act as Adversarial Signatures: The "slice" of an adversarial example looks fundamentally different from a normal example, even if the final output label is identical. Because adversarial attacks rely on unnatural, invisible perturbations to trick the network, they force the data down bizarre computational paths.
  • Targeted Pruning and Protection: You don't always need a model to recognize all 1,000 ImageNet classes. NNSlicer allows developers to slice a model for a specific subset of classes (e.g., just recognizing "dogs"). You can then prune the rest to save space, or heavily encrypt just that specific slice to protect your IP while saving on computational overhead.

Key Results

  • Adversarial Detection: By training a simple decision tree on the slices generated by NNSlicer, the system achieved a 100% recall and 83% precision on average across 17 different attack types. It vastly outperformed baseline methods that relied on raw feature maps or static path extraction.
  • Targeted Model Pruning: NNSlicer can prune 55% of a network's weights while maintaining ~80% accuracy on the targeted classes without any fine-tuning. With just 1 epoch of fine-tuning, the sliced model recovers almost all of its original accuracy for those specific classes.
  • Model Protection: By hiding just 50% of the parameters selected by NNSlicer, an attacker trying to steal the model via retraining achieves less than 10% accuracy on the target classes, rendering the stolen model useless.
  • Performance: While backward analysis is computationally heavy, batch processing allows NNSlicer to slice a ResNet10 model in about 3.4 seconds per sample.

Limitations and Future Work

  • Architectural Limits: Currently, NNSlicer only supports five common CNN operations (Weighted sum, Average, Maximum, Rectify, Scale). It does not yet support Recurrent Neural Networks (RNNs) or Graph Convolutional Networks (GCNs), though the authors note adding backtracking rules for new operators is straightforward.
  • Scalability: Slicing massive models is slow. For a ResNet18 model, computing a slice for a single input takes nearly 10 minutes without batching (though batching drops this to ~45 seconds).
  • Future Directions: The authors suggest expanding the slicing criteria beyond just output neurons to intermediate neurons (similar to inspecting an intermediate variable in traditional code). They also propose using slices to debug specific discriminatory attributes (e.g., isolating the sub-network responsible for race or gender bias) to improve model fairness.

Summary updated

Abstract

Program slicing has been widely applied in a variety of software engineering tasks. However, existing program slicing techniques only deal with traditional programs that are constructed with instructions and variables, rather than neural networks that are composed of neurons and synapses. In this paper, we introduce NNSlicer, the first approach for slicing deep neural networks based on data-flow analysis. Our method understands the reaction of each neuron to an input based on the difference between its behavior activated by the input and the average behavior over the whole dataset. Then we quantify the neuron contributions to the slicing criterion by recursively backtracking from the output neurons, and calculate the slice as the neurons and the synapses with larger contributions. We demonstrate the usefulness and effectiveness of NNSlicer with three applications, including adversarial input detection, model pruning, and selective model protection. In all applications, NNSlicer significantly outperforms other baselines that do not rely on data flow analysis.
BibTeX
@inproceedings{zhang2020dynamic,
  author = {Ziqi Zhang and Yuanchun Li and Yao Guo and Xiangqun Chen and Yunxin Liu},
  title = {{Dynamic slicing for deep neural networks}},
  booktitle = {ESEC/SIGSOFT FSE},
  year = {2020},
  url = {https://openreview.net/forum?id=KUuSch6CmZ},
}

Loading related content...

Figure