Introduction
Deep Neural Networks (DNNs) have achieved extraordinary performance in domains ranging from medical image analysis to natural language processing. However, their deployment in security-critical applications is hindered by a fundamental flaw: vulnerability to adversarial attacks. These attacks involve adding sophisticated, often imperceptible perturbations to benign inputs, causing the model to misclassify them with high confidence. To build truly trustworthy machine learning systems, we must understand these vulnerabilities through both empirical experimentation and rigorous theoretical analysis.
This research addresses the "Trustworthy Machine Learning" gap by developing novel methods to both break and verify DNNs. The work spans interpretable digital attacks, robust physical-world evasions, and high-speed automated verification frameworks.
Empirical Investigations: Exposing Vulnerabilities
Traditional adversarial attacks often perturb every pixel in an image, making it difficult to discern why a model failed. This dissertation introduces StrAttack (Structured Attack), a framework that explores group sparsity in perturbations. By using a sliding mask and the Alternating Direction Method of Multipliers (ADMM), StrAttack identifies the "minimally sufficient" spatial structures required to cause misclassification. This approach bridges the gap between attacking a model and explaining its internal logic, revealing the specific features—often irrelevant textures—the AI over-relies on.
Moving beyond the digital realm, the research tackles the challenge of Physical World Fragility. Most digital attacks fail in the real world due to lighting shifts, distance variations, and object deformation. To address this, the author developed the Adversarial T-shirt. Unlike rigid adversarial patches, this T-shirt is designed to evade person detectors (like YOLOv2) even as the fabric wrinkles and folds on a moving person. This was achieved by combining Expectation over Transformation (EoT) with Thin Plate Spline (TPS) mapping to model non-rigid deformations. In physical testing, this method achieved a 57% success rate, significantly outperforming the 18% success rate of standard adversarial patches.
Theoretical Foundations: Proving Safety
While empirical attacks expose flaws, theoretical frameworks are necessary to guarantee safety. The research extends robustness to Graph Neural Networks (GNNs), which are used in social networks and chemical modeling. Because graph structures are discrete (edges either exist or they do not), the author utilized convex relaxation to enable gradient-based adversarial training, creating the first optimization-based defense for GNNs.
At the core of the dissertation's theoretical contribution is LiRPA (Linear Relaxation based Perturbation Analysis). LiRPA is an automated framework that computes provable linear bounds for neural network outputs under input perturbations. Much like automatic differentiation simplified backpropagation, LiRPA simplifies the process of calculating safety bounds for complex architectures.
To make these theoretical bounds practical, the research introduces Fast and Complete Verification. Historically, proving a model's safety was an NP-hard problem restricted to small models due to the slow nature of CPU-based Linear Programming (LP) solvers. By formulating LiRPA bounds as matrix operations, the author leveraged the parallel processing power of GPUs. Combined with a Branch and Bound (BaB) algorithm, this framework accelerates verification by 10x to 30x over previous state-of-the-art methods, enabling the verification of larger models like WideResNet and datasets like Tiny ImageNet for the first time.
Conclusion and Future Outlook
The path to trustworthy AI requires a dual focus on understanding how models break and proving when they are safe. This research demonstrates that sparsity can lead to interpretability, physical deformations can be modeled mathematically, and formal verification can be scaled through GPU acceleration.
Future work will focus on Patch Transformers to better model environmental noise and the development of real-time verification systems. The ultimate goal is to provide safety guarantees in milliseconds, a requirement for the safe integration of AI into autonomous vehicles and other high-stakes, real-time environments.
Summary updated