Empirical Quantification of Spurious Correlations in Malware Detection

Bianca Perasso, Ludovico Lozza, Andrea Ponte, Luca Demetrio, Luca Oneto, Fabio Roli

TL;DR

End-to-end deep learning models for malware detection are notoriously opaque, often relying on "spurious correlations" like metadata and compiler artifacts rather than actual executable code. This paper introduces a framework using Integrated Gradients to empirically quantify this reliance, revealing that popular models like MalConv and BBDNN dedicate a surprising amount of attention to non-semantic file regions like "slack bytes."

What Problem Does This Paper Address?

End-to-end deep learning models process raw binary bytes directly to detect malware, bypassing traditional, error-prone feature engineering. While highly effective, these models operate as black boxes. Security researchers have long suspected that these networks "cheat" by focusing on metadata or structural anomalies (spurious correlations) rather than the actual malicious payload.

However, prior work has failed to quantify how much these models rely on these irrelevant regions. This paper fills that gap by measuring how much attention models pay to non-executable parts of Windows Portable Executable (PE) files versus the actual compiled code, helping practitioners understand if a model is truly robust or just exploiting fragile artifacts.

Methods

The authors present a method to map and quantify feature importance across different structural regions of a Windows binary:

  1. Structural Segmentation: Using domain knowledge of the Windows PE format (illustrated in Figure 2), they divide files into semantically distinct regions: the DOS header, slack bytes (uninitialized padding inserted by compilers for alignment), the overlay (extra data appended to the end of the file), and the .text section (the actual executable code).
  2. Feature Attribution: They use Integrated Gradients (IG), a gradient-based explainability method, to calculate an importance score for every single byte in a file during model inference.
  3. Quantification (Algorithm 1): They compute the $\ell_2$ norm of the attributions within each isolated region and normalize it against the total attribution. Finally, they calculate an Aggregate Score by subtracting the spurious region attributions from the code section attribution. A positive score means the model prioritizes code; a negative score means it prioritizes noise.
Figure 1: Recap of our analysis: given an end-to-end model and Windows programs, we compute Integrated Gradients, and we extract the attributed relevance of (i) the DOS header, (ii) the slack bytes, (iii) the overlay, and (iv) the “.text” section. These are aggregated to obtain a score, which hint how much the input model relies on spurious correlations when computing predictions.

Datasets

The evaluation uses a balanced dataset of 420 Windows PE files:

  • 210 Malware Samples: Sourced from the Speakeasy testset, evenly distributed (30 each) across seven families: Backdoor, Coinminer, Dropper, Keylogger, Ransomware, RAT, and Trojan.
  • 210 Goodware Samples: Collected from a clean installation of Windows 11 (specifically the System32 folder).

The framework evaluates two well-known pre-trained models: MalConv (which truncates inputs to 1MB) and BBDNN (which truncates inputs to 100KB), both originally trained on the EMBER dataset.

Key Insights and Ideas

  • Models Love "Slack Bytes": Compilers leave empty padding bytes between sections to satisfy alignment requirements. Deep learning models pay an alarming amount of attention to these empty spaces, particularly when evaluating benign software (goodware).
  • Goodware vs. Malware Divergence: For goodware, models rely heavily on slack bytes (up to 40% of attribution for BBDNN). For malware, this reliance drops significantly. This suggests that malware authors' obfuscation or custom compilation techniques alter these padding regions, inadvertently tipping off the model.
  • The Truncation Blindspot: The overlay region (appended data at the end of a file) showed zero attribution across the board. This isn't because overlays are irrelevant, but because both MalConv and BBDNN truncate files to fit their strict input size limits, completely cutting off the overlay data for larger files.

Key Results

The quantitative breakdown shows that while both models maintain a positive Aggregate Score—meaning they still favor actual code over noise—the margin is slim.

  • MalConv achieved an aggregate score of 0.2939. While its code relevance sat around 0.51–0.54, slack bytes captured a significant 0.2567 of the attribution in goodware.
  • BBDNN scored 0.2502. Crucially, slack bytes absorbed a massive 0.4082 of the attribution for goodware samples, nearly rivaling the actual code section (0.4802).
Average attribution of Integrated Gradients on MalConv, showing peaks outside the .text section.

Limitations and Future Work

  • Scale: The study was conducted on a relatively small dataset (420 files). The authors plan to scale this up to the entire Speakeasy dataset to verify if these trends hold.
  • Input Window Constraints: The strict byte-truncation limits of MalConv and BBDNN artificially zeroed out the relevance of overlays. Future work needs to account for variable or larger input windows.
  • Static vs. Dynamic: This study only looks at static analysis (raw bytes). The authors note that spurious correlations likely exist in dynamic analysis (behavioral logs) as well, which they plan to investigate next.

Summary updated

Abstract

End-to-end deep learning exhibits unmatched performance for detecting malware, but such an achievement is reached by exploiting spurious correlations -- features with high relevance at inference time, but known to be useless through domain knowledge. While previous work highlighted that deep networks mainly focus on metadata, none investigated the phenomenon further, without quantifying their impact on the decision. In this work, we deepen our understanding of how spurious correlation affects deep learning for malware detection by highlighting how much models rely on empty spaces left by the compiler, which diminishes the relevance of the compiled code. Through our seminal analysis on a small-scale balanced dataset, we introduce a ranking of two end-to-end models to better understand which is more suitable to be put in production.
BibTeX
@inproceedings{perasso2025empirical,
  author = {Bianca Perasso and Ludovico Lozza and Andrea Ponte and Luca Demetrio and Luca Oneto and Fabio Roli},
  title = {{Empirical Quantification of Spurious Correlations in Malware Detection}},
  booktitle = {Ital-IA},
  year = {2025},
}
Figure