Perceptual Hash of Neural Networks

Zhiying Zhu, Hang Zhou, Siyuan Xing, Zhenxing Qian, Sheng Li, Xinpeng Zhang

TL;DR

This paper introduces HNN-Net, a novel method for generating perceptual hashes of deep neural networks (DNNs). By representing DNNs as graphs and using graph neural networks, HNN-Net creates compact binary codes that capture both the structural and semantic similarities between models, enabling efficient retrieval and protection against model theft.

What Problem Does This Paper Address?

The proliferation of deep learning has led to a massive increase in the development and deployment of DNN models. This abundance, coupled with the high cost of training these models, makes them attractive targets for intellectual property theft through direct copying or knowledge distillation. Existing methods for identifying similar models are often computationally expensive or not well-suited for the diverse landscape of DNN architectures and frameworks. The paper addresses the critical need for an efficient and robust mechanism to retrieve similar DNN models, thereby aiding in the detection of pirated or unauthorized copies.

Methods

HNN-Net employs a two-stage approach:

  1. Graph Generator: This stage converts a raw DNN model into a structured computational graph. The process involves:
  2. ONNX Conversion: Standardizing the model into the ONNX (Open Neural Network Exchange) format to ensure compatibility across various deep learning frameworks (TensorFlow, PyTorch, etc.).
  3. Merge Operation: Applying predefined rules to merge common sub-structures (e.g., Conv-ReLU layers) into single nodes. This aims to simplify the graph while retaining essential information and leveraging "transcendental knowledge" embedded in these common patterns.
  4. Node Feature Embedding: Extracting crucial information like FLOPs and parameter counts for each layer and encoding them along with operator types into a unified node representation.
  5. Graph Hashing: This stage takes the generated graph and learns to produce a compact, Q-bit binary hash code. It utilizes a graph neural network (GNN) architecture, specifically incorporating:
  6. Graph Convolutional Networks (GCNs): To aggregate information from neighboring nodes and learn node-level representations.
  7. Graph Attention Pooling (GAP): To generate a graph-level embedding from the node representations.
  8. Fully-Connected Layers and Binarization: To process the graph embedding and ultimately output the binary hash code.

The training process uses a pair-wise loss function that combines three components: topology loss (preserving structural similarity), classification loss (discriminating between model types), and quantization loss (minimizing information loss during binarization).

Datasets

The paper introduces a new DNN model dataset consisting of 22 models from 10 categories, extracted from the PyTorch "torchvision.models" package. For data augmentation, 10% of nodes and their edges were randomly deleted from the computational graphs. The dataset was split into 80% for training and 20% for testing.

Key Insights and Ideas

  • DNNs as Graphs: The core idea of representing complex DNN architectures as computational graphs is a powerful abstraction that allows for the application of graph-based learning techniques.
  • Perceptual Hashing for Models: Adapting the concept of perceptual hashing, traditionally used for images, to DNN models is a novel approach for measuring model similarity. This allows for "fuzzy matching" rather than exact matches, which is crucial for detecting variations or slightly modified pirated models.
  • Leveraging Structure and Semantics: HNN-Net's success hinges on its ability to capture both the structural layout (topology) and the functional behavior (semantics) of DNNs, providing a more comprehensive representation than purely structural methods.
  • Framework Agnosticism: By using ONNX as an intermediate representation, HNN-Net achieves compatibility with a wide range of deep learning frameworks, making it broadly applicable.

Key Results

  • HNN-Net significantly outperforms baseline methods like GHashing and a simplified version of HNN-Net (without classification loss) in terms of recall, precision, and F1 score, especially when using larger hash code sizes (Q > 16).
  • The proposed method demonstrates good retrieval performance across different hash code lengths (16, 32, 64 bits).
  • The merge operation in the graph generator significantly improves performance, highlighting the value of capturing common structural patterns.
  • The classification loss component is shown to be crucial for enhancing discriminative power and overall performance.

Limitations and Future Work

  • Limited Deep Information Usage: The authors acknowledge that HNN-Net primarily utilizes "shallow" information (topology, node structure, operation details) and has limited access to the "deep" functional information of the network itself.
  • Future Directions: The authors suggest combining shallow and deep information of DNN models to generate even better quality perceptual hash codes. This could involve exploring more sophisticated ways to extract and encode the functional behavior of network layers.

Summary updated

Abstract

In recent years, advances in deep learning have boosted the practical development, distribution and implementation of deep neural networks (DNNs). The concept of symmetry is often adopted in a deep neural network to construct an efficient network structure tailored for a specific task, such as the classic encoder-decoder structure. Massive DNN models are diverse in category, quantity and open source frameworks for implementation. Therefore, the retrieval of DNN models has become a problem worthy of attention. To this end, we propose a new idea of generating perceptual hashes of DNN models, named HNN-Net (Hash Neural Network), to index similar DNN models by similar hash codes. The proposed HNN-Net is based on neural graph networks consisting of two stages: the graph generator and the graph hashing. In the graph generator stage, the target DNN model is first converted and optimized into a graph. Then, it is assigned with additional information extracted from the execution of the original model. In the graph hashing stage, it learns to construct a compact binary hash code. The constructed hash function can well preserve the features of both the topology structure and the semantics information of a neural network model. Experimental results demonstrate that the proposed scheme is effective to represent a neural network with a short hash code, and it is generalizable and efficient on different models.
BibTeX
@inproceedings{zhu2022perceptual,
  author = {Zhiying Zhu and Hang Zhou and Siyuan Xing and Zhenxing Qian and Sheng Li and Xinpeng Zhang},
  title = {{Perceptual Hash of Neural Networks}},
  booktitle = {Symmetry},
  year = {2022},
  url = {https://openreview.net/forum?id=eu2CGzadGav},
}
Figure