A comprehensive study on learning-based PE malware family classification methods

Yixuan Ma, Shuang Liu, Jiajun Jiang, Guanhong Chen, Keqiu Li

Abstract

This study presents a systematic evaluation of nine state-of-the-art deep learning methods for Portable Executable (PE) malware family classification. We investigate three primary input formats: image-based, binary-based, and disassembly-based. Our findings reveal that while models achieve high accuracy (>95% F1-score) on static benchmarks, they experience a significant performance collapse (averaging 32.23%) when subjected to concept drift. Furthermore, we identify a substantial disconnect between academic model performance and industrial deployment requirements regarding latency and memory overhead.

Introduction

Malware classification is a critical component of modern cybersecurity defenses. However, the phenomenon of concept drift—where malware authors evolve their code to evade detection—poses a significant challenge to the long-term efficacy of deep learning models. While many research papers claim near-perfect accuracy, they often rely on stale data and inconsistent experimental setups. This paper addresses this gap by systematically evaluating representative models to determine which input formats are most robust and how they align with real-world industry constraints.

Methodology

The researchers categorized nine representative models into three mainstream approaches based on their input format:

  1. Image-based: Converting binary files into 2D images and utilizing Computer Vision architectures such as ResNet-50, VGG-16, and Inception-V3.
  2. Binary-based: Treating raw bytes as sequences, processed by models like MalConv (1D CNN) or CBOW+MLP.
  3. Disassembly-based: Decompiling malware into assembly code to analyze opcode sequences or Control Flow Graphs (CFGs) using models like MAGIC or MCSC.
Table 1: Categorization of evaluated models, including input formats and evaluation datasets.

The study also explored "aggressive" transfer learning for image models. It was found that fine-tuning convolutional layers, rather than just the final classification layer, significantly improved the model's ability to capture malware-specific features.

Datasets

To ensure a comprehensive evaluation, the study utilized four distinct datasets:

  • BIG-15 & Malimg: Standard benchmarks used in prior research.
  • MalwareBazaar: A new dataset of 3,971 recent PE samples (from 2020) to reflect modern trends.
  • MalwareDrift: A specialized dataset designed to measure concept drift by splitting families into "pre-drift" and "post-drift" samples based on code evolution spikes.

Results and Discussion

Evaluation across the datasets revealed that no single category consistently outperforms the others; performance is more dependent on specific architecture than the input format. While models like CBOW+MLP demonstrate high consistency on static data, they are often the most fragile under concept drift, with CBOW+MLP seeing a staggering 69.62% performance drop.

Table 2: Comprehensive comparison of classification performance and resource overhead across different models and datasets.

A critical "Industry Gap" was identified through interviews with industry partners. Production environments require processing times under 0.1s and memory usage below 1GB. As shown in Table 2, many high-performing models fail to meet these criteria. For instance, MalConv requires over 250GB of RAM in certain training configurations, and disassembly-based methods involve lengthy pre-processing times (up to 17 seconds per sample), making them impractical for high-traffic network gateways.

Conclusion

The study highlights the fragility of current malware classification models against temporal evolution. Disassembly-based methods, which focus on underlying functional logic (opcodes), proved more resilient to drift than those relying on raw bytes or pixels. Future research must prioritize the development of lightweight, drift-aware architectures. Additionally, there is a clear need to shift focus toward behavior-based classification (e.g., identifying Ransomware vs. Trojans) to better serve industrial needs.

Summary updated

BibTeX
@inproceedings{ma2021comprehensive,
  author = {Yixuan Ma and Shuang Liu and Jiajun Jiang and Guanhong Chen and Keqiu Li},
  title = {{A comprehensive study on learning-based PE malware family classification methods}},
  booktitle = {ESEC/SIGSOFT FSE},
  year = {2021},
  doi = {10.1145/3468264.3473925},
}
Figure