Adversarial Examples for CNN-Based Malware Detectors

Bingcai Chen, Zhongru Ren, Chao Yu, Iftikhar Hussain, Jintao LiuIEEE Access 2019 · OpenReview

Abstract

The convolutional neural network (CNN)-based models have achieved tremendous breakthroughs in many end-to-end applications, such as image identification, text classification, and speech recognition. By replicating these successes to the field of malware detection, several CNN-based malware detectors have achieved encouraging performance without significant feature engineering effort in recent years. Unfortunately, by analyzing their robustness using gradient-based algorithms, several studies have shown that some of these malware detectors are vulnerable to the evasion attacks (also known as adversarial examples). However, the existing attack methods can only achieve quite low attack success rates. In this paper, we propose two novel white-box methods and one novel black-box method to attack a recently proposed malware detector. By incorporating the gradient-based algorithm, one of our white-box methods can achieve a success rate of over 99%. Without prior knowledge of the exact structure and internal parameters of the detector, the proposed black-box method can also achieve a success rate of over 70%. In addition, we consider adversarial training as a defensive mechanism in order to resist evasion attacks. While proving the effectiveness of adversarial training, we also analyze its security risk, that is, a large number of adversarial examples can poison the training dataset of the detector. Therefore, we propose a pre-detection mechanism to reject adversarial examples. The experiments show that this mechanism can effectively improve the safety and efficiency of malware detection.

Introduction

The rapid evolution of malware poses a significant threat to global cybersecurity. Traditional detection methods, which rely on manually crafted signatures or expert-defined heuristics, are increasingly inadequate against the sheer volume and variety of modern malware variants. Consequently, researchers have turned to deep learning, specifically Convolutional Neural Networks (CNNs), to automate feature extraction and improve detection accuracy. Models such as MalConv process raw bytes directly, achieving impressive performance without the need for complex feature engineering.

However, CNNs are notoriously susceptible to adversarial examples—inputs specifically designed to fool a model into making a wrong classification. While adversarial attacks are well-documented in the image domain, attacking malware detectors is more challenging because modifications must not break the program's functionality. This paper explores the robustness of CNN-based malware detectors and proposes novel attack and defense mechanisms that significantly outperform existing methods.

The CNN Architecture for Malware Detection

The target of this study is a state-of-the-art CNN-based detector designed for Windows Portable Executable (PE) files. As shown in the architecture diagram below, the model processes files up to 2MB. The raw bytes are tokenized and padded before being passed through an 8D embedding layer. The network utilizes a gated convolutional structure, where one branch applies a standard Conv1D and the other applies Conv1D with a Sigmoid activation. The outputs are combined via a gating mechanism, followed by global max-pooling and fully connected layers, ultimately producing a classification via Softmax.

Architecture of the CNN-based malware detector used in the study, featuring gated convolutions and global max-pooling.

Proposed Adversarial Attacks

The researchers focused on attacking the MalConv architecture using two main strategies that prioritize maintaining the malware's functionality while maximizing the evasion rate.

White-box Attacks: Saliency and Gradients

Using a technique called Grad-CAM, the authors generate "saliency vectors" to identify which parts of a file the model perceives as "benign" or "malicious." * Benign Features Append (BFA): This method identifies code blocks from clean files that the model strongly associates with safety and appends them to the end of a malware file (the "overlay" area). * Enhanced-BFA: This takes BFA further by using the Fast Gradient Sign Method (FGSM). Instead of just appending static blocks, it iteratively tweaks the bytes within those blocks to maximize the model's confusion. This method avoids the pitfalls of random noise, which is often filtered out by the CNN's max-pooling layers.

Black-box Attack: Experience-Based

For scenarios where the attacker lacks access to the model's internal parameters, the authors developed an "Experience-based" method. By observing which benign blocks successfully fooled the model in previous attempts, they create a "contribution degree" database. This allows the attacker to craft highly effective payloads without any internal model access.

Defense Strategy: Pre-detection

While Adversarial Training (training the model on adversarial examples) is a common defense, it carries a "poisoning" risk. If too many adversarial examples are added to the training set, the model may begin to misclassify essential system files as malware.

To counter this, the authors propose a pre-detection mechanism that hashes the executable sections of a file. Since most current adversarial attacks rely on appending data to the file's end while keeping the core malicious code intact, this system can spot the malicious core by its hash before the CNN even processes it. This mechanism is remarkably efficient; while a CNN might take nearly an hour to scan 800 files, the hash-based pre-detection takes less than 10 seconds.

Key Results and Insights

  • Attack Success: The Enhanced-BFA white-box attack achieved a staggering 99% success rate by appending just 40 benign blocks (roughly 20KB). The black-box experience-based method achieved a 73% success rate, significantly outperforming standard random attacks.
  • Defense Effectiveness: The proposed pre-detection mechanism was able to reject 100% of the adversarial examples tested, effectively shielding the CNN from being fooled.
  • Functionality: By only modifying the overlay area of the PE files, all generated adversarial examples remained fully functional malware.

Conclusion

This research underscores the vulnerability of modern deep learning malware detectors. While CNNs offer powerful automated detection, they must be paired with robust defense layers. The proposed pre-detection mechanism provides a fast, effective shield against common evasion techniques, suggesting that the future of malware defense lies in hybrid systems that combine the flexibility of AI with the reliability of traditional verification methods.

Summary updated

Abstract

The convolutional neural network (CNN)-based models have achieved tremendous breakthroughs in many end-to-end applications, such as image identification, text classification, and speech recognition. By replicating these successes to the field of malware detection, several CNN-based malware detectors have achieved encouraging performance without significant feature engineering effort in recent years. Unfortunately, by analyzing their robustness using gradient-based algorithms, several studies have shown that some of these malware detectors are vulnerable to the evasion attacks (also known as adversarial examples). However, the existing attack methods can only achieve quite low attack success rates. In this paper, we propose two novel white-box methods and one novel black-box method to attack a recently proposed malware detector. By incorporating the gradient-based algorithm, one of our white-box methods can achieve a success rate of over 99%. Without prior knowledge of the exact structure and internal parameters of the detector, the proposed black-box method can also achieve a success rate of over 70%. In addition, we consider adversarial training as a defensive mechanism in order to resist evasion attacks. While proving the effectiveness of adversarial training, we also analyze its security risk, that is, a large number of adversarial examples can poison the training dataset of the detector. Therefore, we propose a pre-detection mechanism to reject adversarial examples. The experiments show that this mechanism can effectively improve the safety and efficiency of malware detection.
BibTeX
@inproceedings{chen2019adversarial,
  author = {Bingcai Chen and Zhongru Ren and Chao Yu and Iftikhar Hussain and Jintao Liu},
  title = {{Adversarial Examples for CNN-Based Malware Detectors}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=85AH0oFZ5s},
}
Figure