Towards Building an Intelligent Anti-Malware System: A Deep Learning Approach using Support Vector Machine (SVM) for Malware Classification

Abien Fred AgaraparXiv 2017 · DOI

TL;DR

This paper proposes a novel anti-malware system that leverages deep learning (DL) models combined with Support Vector Machines (SVMs) for malware classification. By visualizing malware binaries as images and training CNN, GRU, and MLP models with an L2-SVM classifier, they achieve promising results, with the GRU-SVM model demonstrating the highest predictive accuracy of approximately 84.92%.

What Problem Does This Paper Address?

The ever-evolving threat of malware necessitates robust and intelligent anti-malware systems. Traditional signature-based detection methods struggle with new and unknown malware variants, as they rely on pre-existing databases. The authors aim to address this by developing a system that can generalize and identify new malware through mathematical pattern recognition, effectively learning the relationship between a malware sample and its family.

Methods

The core of this research lies in combining deep learning architectures with a Support Vector Machine (SVM) as the final classification layer. Specifically, they explored three DL models:

  • Convolutional Neural Network (CNN-SVM): Well-suited for image-based tasks, CNNs are used to extract features from the visualized malware.
  • Gated Recurrent Unit (GRU-SVM): GRUs are a type of recurrent neural network adept at handling sequential data, which the authors argue is relevant even for image data derived from binaries.
  • Multilayer Perceptron (MLP-SVM): A standard feedforward neural network architecture.

For all models, the L2-SVM was employed as the classifier, trained using the one-versus-all (OvA) approach for multinomial classification. The authors also detail the preprocessing steps, including resizing malware images and standardizing features.

Datasets

The primary dataset used in this study is the Malimg dataset [12]. This dataset consists of 9,339 malware samples from 25 different malware families, where each malware binary is visualized as a grayscale image. The dataset was split into 70% for training and 30% for testing.

Key Insights and Ideas

  • Visualizing Malware as Images: The novel approach of treating malware binaries as images allows for the application of powerful image-processing DL techniques to malware classification.
  • DL + SVM Synergy: Combining deep learning feature extraction with the discriminative power of SVMs offers a robust classification framework. The SVM acts as a more stable and potentially more effective classifier than traditional softmax, especially when dealing with complex feature spaces.
  • GRU's Potential for Malware: The GRU-SVM model's superior performance suggests that the sequential nature of data, even when visualized, can be effectively leveraged by recurrent architectures for malware analysis. This hints at the underlying structure within malware binaries that GRUs can capture.
  • Architecture Matters: The paper strongly suggests a correlation between the complexity of the DL architecture (number of layers, gating mechanisms) and its predictive performance. More sophisticated architectures, like the GRU, tend to yield better results.

Key Results

  • GRU-SVM Dominance: The GRU-SVM model achieved the highest test accuracy of approximately 84.92%.
  • Comparative Performance: The MLP-SVM model followed with an accuracy of 80.47%, and the CNN-SVM model achieved 77.23%.
  • Training Time Trade-off: While GRU-SVM performed best, it also had the longest training time (11 minutes and 32 seconds), highlighting a trade-off between performance and computational cost. MLP-SVM was the fastest.
  • Family-Specific Performance: All models performed better on malware families with a higher number of variants, likely due to more data available for learning.

Limitations and Future Work

  • Dataset Partitioning: The authors acknowledge that the relative populations of malware families were not considered during dataset partitioning, which might have influenced performance on smaller families.
  • Optimization of CNN/MLP: The paper suggests that improving the architecture of CNN-SVM and MLP-SVM models by adding more hidden layers, better non-linearities, or optimized dropout could potentially boost their performance to match or exceed the GRU-SVM.
  • Further Exploration: The authors propose that exploring more optimal DL-SVM models is the next logical step towards building a truly intelligent anti-malware system. This could involve investigating different DL architectures, hyperparameter tuning, and potentially larger or more diverse datasets.

Summary updated

Abstract

Effective and efficient mitigation of malware is a long-time endeavor in the information security community. The development of an anti-malware system that can counteract an unknown malware is a prolific activity that may benefit several sectors. We envision an intelligent anti-malware system that utilizes the power of deep learning (DL) models. Using such models would enable the detection of newly-released malware through mathematical generalization. That is, finding the relationship between a given malware $x$ and its corresponding malware family $y$, $f: x \mapsto y$. To accomplish this feat, we used the Malimg dataset (Nataraj et al., 2011) which consists of malware images that were processed from malware binaries, and then we trained the following DL models 1 to classify each malware family: CNN-SVM (Tang, 2013), GRU-SVM (Agarap, 2017), and MLP-SVM. Empirical evidence has shown that the GRU-SVM stands out among the DL models with a predictive accuracy of ~84.92%. This stands to reason for the mentioned model had the relatively most sophisticated architecture design among the presented models. The exploration of an even more optimal DL-SVM model is the next stage towards the engineering of an intelligent anti-malware system.
BibTeX
@misc{agarap2017towards,
  author = {Abien Fred Agarap},
  title = {{Towards Building an Intelligent Anti-Malware System: A Deep Learning Approach using Support Vector Machine (SVM) for Malware Classification}},
  year = {2017},
  eprint = {1801.00318},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1801.00318},
}
Figure