MFDroid: A Stacking Ensemble Learning Framework for Android Malware Detection

Xusheng Wang, Lin‐Lin Zhang, Kai Zhao, Xuhui Ding, Mingming YuSensors 2022 · DOI

TL;DR

This paper introduces MFDroid, a novel stacking ensemble learning framework for Android malware detection. By combining multiple feature selection algorithms and diverse base learners, MFDroid achieves a high F1-score of 96.0%, outperforming single models and demonstrating the limitations of relying solely on permissions for malware detection.

What Problem Does This Paper Address?

The proliferation of Android devices has led to a surge in Android malware, posing significant threats to user privacy and security. Traditional machine learning approaches for malware detection often struggle with the complexity and evolving nature of these threats. Specifically, the paper highlights two key challenges:

  1. Limited effectiveness of single feature selection algorithms: These algorithms can miss crucial features, leading to suboptimal detection performance.
  2. Low detection efficiency of traditional single machine learning models: These models can be sensitive to data skew, computational overhead, and may not generalize well to diverse malware behaviors.

The authors also note a critical trend: malicious and benign applications are becoming increasingly similar in their permission requests, rendering permission-based detection alone insufficient.

Methods

MFDroid employs a two-stage stacking ensemble learning approach to address these challenges. The framework consists of three main stages:

  1. Feature Preprocessing: APK files are analyzed using tools like AndroPyTool to extract static features, including permissions, API calls, and opcodes. These features are then vectorized.
  2. Feature Selection: A novel approach is used where the results from seven different feature selection algorithms (Decision Tree, Gradient Boosting Decision Tree, Extra-Trees, Chi-square test, Genetic Algorithm, SVM-Recursive Feature Elimination, and Mutual Information) are combined using a union strategy. This aims to capture a comprehensive set of relevant features, mitigating the limitations of individual algorithms.
  3. Stacking Ensemble Method:
    • Base Learners (First Layer): Five diverse machine learning models (SVM, GBDT, XGBoost, LightGBM, and CatBoost) are trained on the combined feature set using K-fold cross-validation.
  4. Meta-Classifier (Second Layer): Logistic Regression is used to learn from the predictions of the base learners, producing the final classification.

This ensemble approach is designed to leverage the strengths of multiple models and feature sets, leading to more robust and accurate malware detection.

Datasets

The study utilizes two datasets:

  1. A newly constructed real-world Android malware dataset: This dataset comprises 1664 Android applications collected from Chinese app markets between 2019 and 2020, consisting of 806 malicious apps and 858 benign apps across 15 categories. Malware was identified using the VirusTotal web tool.
  2. The OmniDroid dataset: This dataset is used for comparative analysis with other existing studies.

Key Insights and Ideas

  • Permissions are becoming less discriminative: The study strongly suggests that relying solely on permission requests is no longer an effective strategy for distinguishing malicious from benign Android applications due to increasing similarity.
  • Feature union strategy is superior: Combining features selected by multiple algorithms (union strategy) significantly outperforms individual feature selection methods and even an intersection strategy, highlighting the benefit of a comprehensive feature set.
  • Ensemble learning for robustness: The stacking ensemble approach, by combining diverse base learners and a meta-classifier, effectively mitigates the limitations of individual models and leads to improved detection accuracy and stability.
  • Opcodes are important, but not sufficient alone: While opcodes are fundamental to application execution and can be informative, they are not enough on their own to differentiate malware from benign apps, necessitating their combination with other features.

Key Results

  • MFDroid achieved an impressive F1-score of 96.0% on its custom dataset.
  • The union strategy for feature selection resulted in a 2.3% higher F1-score and a 2.4% higher accuracy compared to the best performing single feature selection algorithm (GA).
  • MFDroid demonstrated superior performance compared to individual base learners (SVM, GBDT, XGBoost, LightGBM, CatBoost), with its F1-score being 4.9% to 7.4% higher and its accuracy being 3.1% to 7.9% higher.
  • On the OmniDroid dataset, MFDroid also outperformed other comparative methods, achieving an F1-score of 0.960 and an accuracy of 0.959.

Limitations and Future Work

  • Limitations of single feature selection: The paper acknowledges that single feature selection algorithms can miss important features, which is why the union strategy was adopted.
  • Limitations of single machine learning algorithms: Single models are susceptible to data skew, high computational overhead, sensitivity to initial settings, and can struggle with high-dimensional features.
  • Future Work: The authors plan to explore static and dynamic features more comprehensively to describe application behavior, aiming to improve detection accuracy by incorporating a wider range of features.

Summary updated

Abstract

As Android is a popular a mobile operating system, Android malware is on the rise, which poses a great threat to user privacy and security. Considering the poor detection effects of the single feature selection algorithm and the low detection efficiency of traditional machine learning methods, we propose an Android malware detection framework based on stacking ensemble learning-MFDroid-to identify Android malware. In this paper, we used seven feature selection algorithms to select permissions, API calls, and opcodes, and then merged the results of each feature selection algorithm to obtain a new feature set. Subsequently, we used this to train the base learner, and set the logical regression as a meta-classifier, to learn the implicit information from the output of base learners and obtain the classification results. After the evaluation, the F1-score of MFDroid reached 96.0%. Finally, we analyzed each type of feature to identify the differences between malicious and benign applications. At the end of this paper, we present some general conclusions. In recent years, malicious applications and benign applications have been similar in terms of permission requests. In other words, the model of training, only with permission, can no longer effectively or efficiently distinguish malicious applications from benign applications.
BibTeX
@inproceedings{wang2022mfdroid,
  author = {Xusheng Wang and Lin‐Lin Zhang and Kai Zhao and Xuhui Ding and Mingming Yu},
  title = {{MFDroid: A Stacking Ensemble Learning Framework for Android Malware Detection}},
  booktitle = {Sensors},
  year = {2022},
  doi = {10.3390/s22072597},
}
Figure