Towards Trustworthy Deep Learning

Tsui-Wei Lily WengAAAI 2024 · DOI

Introduction

Deep Neural Networks (DNNs) have become the backbone of modern artificial intelligence, powering everything from autonomous vehicles to medical diagnostic tools. However, as these models are integrated into mission-critical systems, two fundamental flaws persist: vulnerability and opacity. DNNs are often "black boxes" that can be easily fooled by adversarial perturbations and are notoriously difficult for humans to interpret. Dr. Tsui-Wei (Lily) Weng’s research addresses these challenges by developing automated, scalable tools designed to certify neural network robustness and illuminate the internal logic of deep learning models.

The Challenge: Vulnerability and Opacity

The deployment of AI in high-stakes environments is hindered by two primary hurdles. First, DNNs are susceptible to "adversarial examples"—small, often imperceptible changes to input data that lead to catastrophic failures. Traditional defense mechanisms are frequently reactive, addressing specific attacks rather than providing general security.

Second, understanding the "why" behind a model's decision has historically been a manual, labor-intensive process. Researchers typically define concepts (e.g., "wheels" or "fur") and hand-label thousands of images to determine if a specific neuron recognizes those features. This "brute-force" approach is not only expensive but also limits interpretability to a pre-defined, human-centric vocabulary.

Pillar 1: Robust Machine Learning through Provable Guarantees

Dr. Weng’s work moves beyond the "cat-and-mouse" game of adversarial attacks by establishing formal metrics and certification frameworks that provide mathematical guarantees of safety.

  • CLEVER Score: Utilizing extreme value theory, the CLEVER score provides an attack-agnostic metric. It estimates the "safety margin" of a model, offering a statistical guarantee of robustness regardless of the specific attack strategy employed.
  • CNN-Cert: While early certification techniques were limited to simple networks, CNN-Cert generalizes linear bounding techniques. This allows for the certification of complex architectures, including ResNets and LeNets, making formal verification practical for modern vision models.
  • SingleProp: To bridge the gap between theory and practice, SingleProp introduces an efficient training algorithm. By employing novel regularization techniques, it improves model robustness while training up to 13.5x faster than previous certified defense methods.

Pillar 2: Automated and Open-Vocabulary Interpretability

To solve the opacity problem, Dr. Weng’s research focuses on removing the human-in-the-loop bottleneck, allowing for a more transparent understanding of how models process information.

  • CLIP-Dissect: By leveraging foundation models like CLIP (Contrastive Language-Image Pre-training), this method automatically describes neuron functionality using an open vocabulary. This eliminates the need for manual labeling, allowing researchers to understand model internals 10–200x faster than traditional methods.
  • Label-free Concept Bottleneck Models (CBMs): Traditional CBMs require humans to define the intermediate concepts a model uses. Dr. Weng’s approach automates this process, transforming standard NNs into interpretable CBMs without sacrificing accuracy. This work represents the first time such models have successfully scaled to large-scale datasets like ImageNet.

Key Insights and Impact

The core philosophy of this research is the transition from manual to automated trust. By using foundation models to label neurons and statistical theory to measure safety, the "black box" becomes transparent and verifiable at a fraction of the previous cost. Crucially, this work demonstrates that there is no inherent trade-off between accuracy and interpretability; models can perform at state-of-the-art levels while remaining human-understandable.

Future Frontiers

While significant progress has been made in computer vision, the roadmap for trustworthy AI extends into several emerging domains:

  • Large Language Models (LLMs): Extending neuron-level explanations to the massive scale and complexity of transformer architectures.
  • Semantic Robustness: Moving beyond pixel-level noise to certify models against environmental changes like lighting, weather, or rotation.
  • Reinforcement Learning (RL): Ensuring the safety and transparency of agents operating in dynamic, multi-step environments.

Dr. Weng’s research provides the necessary tools to ensure that as AI becomes more powerful, it also becomes more reliable, scalable, and understandable.

Summary updated

BibTeX
@inproceedings{weng2024towards,
  author = {Tsui-Wei Lily Weng},
  title = {{Towards Trustworthy Deep Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i20.30298},
}
Figure