Set-Valued Sensitivity Analysis of Deep Neural Networks

Xin Wang, Feilong Wang, Xuegang (Jeff) BanAAAI 2025 · DOI

TL;DR

This paper introduces a novel "set-to-set" sensitivity analysis framework for Deep Neural Networks (DNNs) that accounts for the fact that DNNs can have multiple valid solutions (minima). By analyzing how the set of possible solutions changes with data perturbations, the authors provide a more robust understanding of DNN training stability and reliability, even when the loss function isn't strictly convex.

What Problem Does This Paper Address?

Traditional sensitivity analysis in machine learning often focuses on how a single output or solution changes when inputs or parameters are slightly altered. However, for Deep Neural Networks (DNNs), the optimization process doesn't always converge to a unique solution. Instead, it can find a "flat minimum" or a whole manifold of solutions. This means that if you retrain a DNN, you might get a different set of weights that still perform well. Existing sensitivity analysis methods, which assume a single solution, overlook this crucial aspect. This can lead to a false sense of security or inaccurate evaluations, especially in critical applications like security or robustness testing, where understanding the full range of possible model behaviors is essential. This paper addresses this gap by developing a framework to analyze the sensitivity of the entire solution set of a DNN to changes in the training data.

Methods

The core innovation of this paper is the shift from a "point-to-point" to a "set-to-set" sensitivity analysis. Instead of tracking how a single weight vector changes, they analyze how the set of all valid weight vectors (the solution set) changes.

Key methodological contributions include:

  • Set-Valued Mapping: The solution of a DNN is framed as a set-valued mapping $S(x)$, where $x$ represents the training data and $S(x)$ is the set of optimal weights.
  • Lipschitz-like Property for Sets: They extend the concept of Lipschitz continuity to set-valued mappings. A DNN exhibits this property if the change in its solution set due to data perturbations can be bounded. This is a key indicator of training stability.
  • Generalized Derivatives: The paper leverages graphical derivatives to analyze the local behavior of these set-valued mappings. This is a more general tool than the implicit function theorem, which often requires non-singular Hessians and thus fails for many DNNs.
  • Estimating New Solution Sets: They propose a method to estimate the new solution set after data perturbations without retraining the entire network. This involves using the graphical derivative to predict the shift and expansion of the solution set.
  • Algorithm for Training Stability: They introduce an algorithm to quantify training stability by measuring the distance between the original and estimated new solution sets.

Datasets

The paper uses the CIFAR-10 dataset for evaluating their method on a ResNet56 network. They also use a synthetic "toy example" with two data points to illustrate their concepts.

Key Insights and Ideas

  • DNNs are inherently "set-valued": The paper emphasizes that the optimization landscape of DNNs often leads to multiple solutions, not just one. Ignoring this can lead to incomplete understanding.
  • "Set-to-set" analysis is more robust: By analyzing the entire solution set, the framework provides a more comprehensive view of DNN behavior under perturbations, especially for non-isolated minima.
  • Lipschitz-like property as a stability metric: Demonstrating this property for DNN solution sets provides theoretical grounding for their stability and robustness against data shifts.
  • Generalized derivatives bypass Hessian limitations: The use of graphical derivatives allows sensitivity analysis even when the Hessian is singular, a common scenario in DNNs.
  • Estimating solution shifts without retraining: This is a significant practical contribution, enabling faster and more efficient analysis of model robustness.

Key Results

  • Theoretical Proof of Lipschitz-like Property: The paper proves that Deep Fully Connected Neural Networks (DFCNNs) with ReLU activation exhibit the Lipschitz-like property for their solution sets.
  • Bound on Lipschitz Modulus: They provide a bound for this Lipschitz modulus, quantifying the expected deviation of the solution set.
  • Empirical Validation: Simulations on a toy example and a ResNet56 network demonstrate that their proposed method can accurately estimate the new solution set after data perturbations. The estimated solutions lead to near-zero training loss on poisoned data, showing the effectiveness of the approach.
  • Comparison to Influence Functions: The paper shows that their graphical derivative-based method is a generalization of influence functions when the solution set is a single point.

Limitations and Future Work

  • Computational Cost: While the method avoids retraining, computing graphical derivatives and related quantities can still be computationally intensive, especially for very large networks.
  • Assumption on Perturbations: The current framework primarily focuses on perturbations of individual data points or the entire dataset. Analyzing more complex perturbation scenarios might be a future direction.
  • Generalization to Other Architectures: While they mention extending to general DNNs and simulate on ResNet56, further rigorous analysis on diverse architectures could be beneficial.
  • Practical Implications for Defense: The authors suggest that their framework can be used to design better defense mechanisms against data poisoning attacks by understanding how solutions shift. This is a promising area for future research.

Summary updated

BibTeX
@inproceedings{wang2025set,
  author = {Xin Wang and Feilong Wang and Xuegang (Jeff) Ban},
  title = {{Set-Valued Sensitivity Analysis of Deep Neural Networks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i20.35430},
}
Figure