Local Repair of Neural Networks Using Optimization

Keyvan Majd, Siyu Zhou, Heni Ben Amor, Georgios Fainekos, Sriram SankaranarayananarXiv 2021 · OpenReview

TL;DR

This paper introduces a novel framework for "repairing" pre-trained neural networks (NNs) to satisfy specific safety or property constraints. By formulating the repair problem as a Mixed-Integer Quadratic Program (MIQP), they adjust the weights of a single NN layer to meet these new requirements while preserving the network's original performance on its training data. This approach offers a practical way to make complex NN systems safer and more reliable.

What Problem Does This Paper Address?

Neural networks (NNs) are increasingly used in critical applications like autonomous driving and robotics. However, these systems can exhibit unsafe behaviors, especially when encountering inputs outside their training distribution. While methods exist to detect these "adversarial samples," fixing the underlying NN to prevent such issues is challenging. Simply retraining with adversarial examples can lead to overfitting or "oscillations" where fixing one problem creates another. Existing repair methods are often limited to specific NN architectures, require overlap between adversarial and training regions, or don't fully account for the impact of repairs on overall performance. This paper addresses the need for a general, effective method to modify NNs to satisfy explicit safety properties without sacrificing their original accuracy.

Methods

The core of this paper is a framework for "layer-wise" NN repair. The authors propose to modify the weights and biases of a single layer of a pre-trained feed-forward NN to satisfy a given predicate (a set of constraints on the NN's output for specific inputs). This repair problem is formulated as a Mixed-Integer Quadratic Program (MIQP).

Key aspects of their method include:

  • Predicate Formulation: Safety or property requirements are expressed as predicates on the NN's output for a target input domain.
  • MIQP Formulation: The problem of adjusting a single layer's weights and biases to satisfy these predicates while minimizing the original loss function (to maintain performance) is cast as an MIQP. This involves handling the non-linear ReLU activation functions using disjunctive inequalities, which are then linearized for the optimization solver.
  • Layer-wise Repair: Instead of modifying the entire network, only one layer is adjusted at a time. This simplifies the optimization problem and allows for targeted repairs.
  • Minimizing Deviation: The MIQP objective function balances minimizing the original loss function with minimizing the deviation of the modified layer's weights from their original values, controlled by a parameter $\delta$.

Datasets

The paper does not introduce new datasets but rather uses existing or synthetically generated data for training and evaluating their repair framework across various applications:

  • In-place Rotation: A two-hidden-layer NN trained on data for a 45-degree counterclockwise rotation in 2D space.
  • Forward Kinematics: A three-hidden-layer NN trained on data generated by the RoboAnalyzer simulator for a 6-DOF KUKA robot arm.
  • Collision Avoidance: A deep neural network (DNN) from the ACAS Xu system, trained for a specific scenario (aprev = WL, τ = 100).
  • Safe Controller: A neural network controller trained via imitation learning to mimic a QP-based controller for a unicycle robot model.

Key Insights and Ideas

  • Targeted Layer Repair is Sufficient: The authors demonstrate that modifying just a single layer can be enough to satisfy complex safety properties, significantly simplifying the repair process compared to modifying the entire network.
  • MIQP as a Powerful Tool: Formulating NN repair as an MIQP provides a principled way to balance conflicting objectives: satisfying new constraints and preserving original performance.
  • Trade-off Between Repair Layer and Performance: Repairing earlier layers (closer to the input) often leads to better performance preservation and error reduction, especially for capturing non-linearities, though it can increase the complexity of the MIQP.
  • Subtle Weight Changes Can Have Large Effects: The paper highlights how even small changes to NN weights can drastically alter its behavior, underscoring the need for careful, optimization-based repair methods.
  • Repairing for Safety vs. Original Performance: The framework explicitly handles cases where safety constraints might require deviating from the original target outputs for specific unsafe input regions.

Key Results

The paper demonstrates the effectiveness of their framework across four distinct applications:

  • In-place Rotation: The repaired network successfully pushed predicted outputs inside a defined quadrilateral safety region, with better prediction accuracy for points inside the region. Repairing earlier layers yielded better results.
  • Forward Kinematics: The repaired network successfully bounded the robot arm's end-effector within a safe operational region, maintaining accurate predictions for points within the bound. Repairing the third layer showed lower error than the fourth.
  • Collision Avoidance: The method successfully corrected local misclassified points in the ACAS Xu system by repairing the last layer, achieving the desired property with minimal weight deviations.
  • Safe Controller: The repaired controller successfully avoided an unsafe ellipsoidal region, with simulated trajectories deviating from the original policy only when near the unsafe area. The optimal weight deviation was small.

Limitations and Future Work

  • Scalability of MIQP: The authors acknowledge that solving MIQPs for high-dimensional robotic systems with large input spaces is computationally challenging.
  • Infeasibility: They note that repairing the last layer did not always satisfy properties and could sometimes lead to infeasibility, though this was not observed when repairing hidden layers.
  • Future Work:
    • Address the scalability issue by relaxing or reducing the number of integer variables in the MIQP formulation.
  • Investigate efficient sampling mechanisms to further reduce computational complexity.
  • Explore cases where infeasibility might occur during repair.
  • Develop a publicly available Python package for NN repair integrated with TensorFlow.

Summary updated

Abstract

In this paper, we propose a framework to repair a pre-trained feed-forward neural network (NN) to satisfy a set of properties. We formulate the properties as a set of predicates that impose constraints on the output of NN over the target input domain. We define the NN repair problem as a Mixed Integer Quadratic Program (MIQP) to adjust the weights of a single layer subject to the given predicates while minimizing the original loss function over the original training domain. We demonstrate the application of our framework in bounding an affine transformation, correcting an erroneous NN in classification, and bounding the inputs of a NN controller.
BibTeX
@misc{majd2021local,
  author = {Keyvan Majd and Siyu Zhou and Heni Ben Amor and Georgios Fainekos and Sriram Sankaranarayanan},
  title = {{Local Repair of Neural Networks Using Optimization}},
  year = {2021},
  eprint = {2109.14041},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2109.14041},
}
Figure