TL;DR
End-to-end deep learning models for malware detection are notoriously opaque, often relying on "spurious correlations" like metadata and compiler artifacts rather than actual executable code. This paper introduces a framework using Integrated Gradients to empirically quantify this reliance, revealing that popular models like MalConv and BBDNN dedicate a surprising amount of attention to non-semantic file regions like "slack bytes."
What Problem Does This Paper Address?
End-to-end deep learning models process raw binary bytes directly to detect malware, bypassing traditional, error-prone feature engineering. While highly effective, these models operate as black boxes. Security researchers have long suspected that these networks "cheat" by focusing on metadata or structural anomalies (spurious correlations) rather than the actual malicious payload.
However, prior work has failed to quantify how much these models rely on these irrelevant regions. This paper fills that gap by measuring how much attention models pay to non-executable parts of Windows Portable Executable (PE) files versus the actual compiled code, helping practitioners understand if a model is truly robust or just exploiting fragile artifacts.
Methods
The authors present a method to map and quantify feature importance across different structural regions of a Windows binary:
- Structural Segmentation: Using domain knowledge of the Windows PE format (illustrated in Figure 2), they divide files into semantically distinct regions: the DOS header, slack bytes (uninitialized padding inserted by compilers for alignment), the overlay (extra data appended to the end of the file), and the
.textsection (the actual executable code). - Feature Attribution: They use Integrated Gradients (IG), a gradient-based explainability method, to calculate an importance score for every single byte in a file during model inference.
- Quantification (Algorithm 1): They compute the $\ell_2$ norm of the attributions within each isolated region and normalize it against the total attribution. Finally, they calculate an Aggregate Score by subtracting the spurious region attributions from the code section attribution. A positive score means the model prioritizes code; a negative score means it prioritizes noise.
Datasets
The evaluation uses a balanced dataset of 420 Windows PE files:
- 210 Malware Samples: Sourced from the Speakeasy testset, evenly distributed (30 each) across seven families: Backdoor, Coinminer, Dropper, Keylogger, Ransomware, RAT, and Trojan.
- 210 Goodware Samples: Collected from a clean installation of Windows 11 (specifically the
System32folder).
The framework evaluates two well-known pre-trained models: MalConv (which truncates inputs to 1MB) and BBDNN (which truncates inputs to 100KB), both originally trained on the EMBER dataset.
Key Insights and Ideas
- Models Love "Slack Bytes": Compilers leave empty padding bytes between sections to satisfy alignment requirements. Deep learning models pay an alarming amount of attention to these empty spaces, particularly when evaluating benign software (goodware).
- Goodware vs. Malware Divergence: For goodware, models rely heavily on slack bytes (up to 40% of attribution for BBDNN). For malware, this reliance drops significantly. This suggests that malware authors' obfuscation or custom compilation techniques alter these padding regions, inadvertently tipping off the model.
- The Truncation Blindspot: The overlay region (appended data at the end of a file) showed zero attribution across the board. This isn't because overlays are irrelevant, but because both MalConv and BBDNN truncate files to fit their strict input size limits, completely cutting off the overlay data for larger files.
Key Results
The quantitative breakdown shows that while both models maintain a positive Aggregate Score—meaning they still favor actual code over noise—the margin is slim.
- MalConv achieved an aggregate score of 0.2939. While its code relevance sat around 0.51–0.54, slack bytes captured a significant 0.2567 of the attribution in goodware.
- BBDNN scored 0.2502. Crucially, slack bytes absorbed a massive 0.4082 of the attribution for goodware samples, nearly rivaling the actual code section (0.4802).
Limitations and Future Work
- Scale: The study was conducted on a relatively small dataset (420 files). The authors plan to scale this up to the entire Speakeasy dataset to verify if these trends hold.
- Input Window Constraints: The strict byte-truncation limits of MalConv and BBDNN artificially zeroed out the relevance of overlays. Future work needs to account for variable or larger input windows.
- Static vs. Dynamic: This study only looks at static analysis (raw bytes). The authors note that spurious correlations likely exist in dynamic analysis (behavioral logs) as well, which they plan to investigate next.
Summary updated