TL;DR
This paper introduces a novel Convolutional Neural Network (CNN) for detecting malware embedded within Hangul Word Processor (HWP) files, a common document format in South Korea. The key innovation lies in a new "stretch padding" technique and a "spatial pyramid average pooling" (SPAP) layer, designed to efficiently handle the highly variable lengths of HWP byte streams, a significant challenge for existing methods.
What Problem Does This Paper Address?
The increasing prevalence of malware in non-executable files, such as HWP documents, poses a significant threat to ordinary users. While executable malware is often well-defended against, non-executable files are more easily overlooked and can be used in targeted attacks. HWP files, in particular, are widely used in South Korea and are a target for malware, often originating from North Korea. Existing malware detection methods for these files often rely on manual feature engineering, which is time-consuming, requires expert knowledge, and struggles to adapt to new malware variants. Furthermore, many deep learning approaches, while capable of automatic feature extraction, are computationally inefficient, making them impractical for analyzing large volumes of suspicious files.
Methods
The authors propose a CNN-based model that directly processes raw byte streams from HWP files. The model incorporates two novel components:
- Stretch Padding: Traditional padding methods (adding special tokens to sequences to equalize length) can lead to information loss, especially with highly variable data like HWP byte streams. "Stretch padding" addresses this by stretching existing byte stream elements and interleaving padding tokens, aiming to preserve more of the original data's semantic information.
- Spatial Pyramid Average Pooling (SPAP): This layer is a variant of Spatial Pyramid Pooling (SPP). Unlike SPP, which operates in two dimensions (like images) and uses max-pooling, SPAP operates in a single dimension (for byte streams) and employs average-pooling. This is crucial because it aggregates semantic information from embedding vectors, not just raw values, allowing for a more nuanced understanding of the byte stream's patterns.
The CNN architecture itself consists of an embedding layer, two convolutional layers, a global max-pooling layer, a fully-connected layer, and an output layer for binary classification (benign/malicious).
Datasets
The study utilizes the HWP dataset from a previous work [3], which contains 534 HWP files (79 benign, 455 malicious). The authors generated benign and malicious stream samples from this dataset with an input length of 100,000 bytes for their experiments.
Key Insights and Ideas
- HWP Byte Streams are Highly Variable: The paper highlights the significant challenge posed by the extremely long and variable lengths of HWP byte streams, with standard deviations in the millions. This variability makes fixed-input-size models or simple padding methods ineffective.
- Average Pooling Captures Deeper Semantics: The authors argue that average pooling in SPAP is more effective than max-pooling for byte streams because it aggregates semantic information from embedding vectors, leading to a better understanding of relational information within the stream.
- Stretch Padding Mitigates Information Loss: The proposed stretch padding method is shown to be more effective than conventional padding, particularly for variable-length sequences, by better preserving important local patterns.
- Efficiency is Crucial for Practical Malware Detection: The paper emphasizes that while accuracy is important, the speed and computational efficiency of a model are critical for real-world malware detection systems that need to process many files quickly.
Key Results
- Effectiveness: The proposed SPAP-Conv model with stretch padding achieved the best F1 scores for both benign and malicious cases, outperforming other models. Specifically, it achieved F1 scores of 92.86/95.08% for benign/malicious cases.
- Efficiency: The SPAP-Conv model is significantly more efficient than existing methods. It has approximately 15 times fewer trainable parameters than Mal-Conv and requires only 0.9831 seconds to process the test set, making it considerably faster than Cons-Conv (15.8662 seconds).
- Comparison: The SPAP-Conv model demonstrated a good balance between effectiveness (F1 score) and efficiency, making it a practical solution for malware detection.
Limitations and Future Work
- Padding Implementation: The authors note that the "slicing" operation used to implement the SPAP layer in TensorFlow is slower than alternatives, suggesting this is an implementation issue rather than a fundamental limitation of the SPAP concept.
- Future Work: The authors suggest exploring deeper SPAP layers for potentially improved performance or shallower structures for enhanced efficiency. They also plan to investigate more malicious HWP files and consider alternative model architectures.
Summary updated