TL;DR
This paper introduces M-Attack, a novel algorithm for generating adversarial examples on mixed-type data (combining numerical and categorical features). M-Attack effectively perturbs both feature types simultaneously to mislead ML models, while a new "Mixed Mahalanobis Distance" helps these adversarial examples remain inconspicuous and evade detection.
What Problem Does This Paper Address?
Machine learning models are increasingly used in safety-critical applications like financial forecasting and fraud detection. However, these models are vulnerable to adversarial attacks, where subtle, imperceptible changes to input data can cause misclassifications. A significant challenge arises when the data itself is "mixed-type," containing both numerical (e.g., age, income) and categorical (e.g., gender, occupation) features. Existing adversarial attack methods typically focus on either numerical or categorical data separately, leaving a gap in how to effectively attack models trained on mixed-type data. Furthermore, adversarial examples need to be "in-distribution" – close to the original data's distribution – to avoid detection by anomaly detection systems or human experts.
Methods
The core of the proposed solution is the M-Attack algorithm. It tackles mixed-type data by:
- Unified Search Space: Transforming the problem into a unified continuous space by representing categorical features as probability distributions. This allows for simultaneous optimization of both numerical and categorical perturbations using gradient-based methods.
- Simultaneous Perturbation: Unlike methods that sequentially attack numerical and categorical features, M-Attack updates both simultaneously, allowing it to better capture their interactions and find more effective adversarial examples.
- Mixed Mahalanobis Distance: To ensure adversarial examples are "in-distribution" and evade detection, the authors introduce a novel Mixed Mahalanobis Distance. This metric measures the distance of an adversarial example to the distribution of clean data, considering the correlations between all mixed-type features. By regularizing this distance, M-Attack generates examples that are less likely to be flagged as outliers.
Datasets
The authors evaluated their method on three real-world datasets:
- IEEE-CIS Transaction Dataset: Used for fraud detection, containing 108 numerical and 32 categorical features.
- Home Credit Default Risk Dataset: Used for loan repayment prediction, with 73 numerical and 28 categorical features.
- Criteo Display Advertising Dataset: Used for click-through rate prediction, featuring 13 numerical and 7 categorical features.
Key Insights and Ideas
- Interactions Matter: The interaction between numerical and categorical features is crucial. Simply attacking them separately (as baselines do) leads to suboptimal adversarial examples. M-Attack's simultaneous update strategy leverages these interactions.
- In-Distribution is Key for Stealth: Adversarial examples that deviate too much from the original data distribution are easily detected. The Mixed Mahalanobis Distance is a clever way to enforce this "in-distribution" property, making attacks more insidious.
- Unified Approach is More Optimal: The paper demonstrates that optimizing for both attack strength (loss maximization) and stealth (low Mixed Mahalanobis Distance) simultaneously, as M-Attack does, yields better results than sequential optimization.
Key Results
- Effectiveness: M-Attack achieved higher successful attack rates compared to baseline methods (combinations of existing numerical and categorical attack techniques) on the CIS Fraud and Criteo datasets.
- Efficiency: M-Attack's computational cost is not significantly affected by the perturbation budget, and it offers competitive efficiency.
- Stealth: Experiments showed that imposing a small Mix Mahalanobis Distance significantly helps generated adversarial examples stay closer to the clean data distribution, making them harder to detect by OOD methods.
- Optimality: When comparing trade-offs between attack loss and Mixed Mahalanobis Distance, M-Attack consistently achieved higher average loss values for similar levels of stealth compared to baselines.
Limitations and Future Work
- Classifier Assumption: The current work assumes a white-box setting, where the attacker has full knowledge of the model's parameters and architecture.
- Future Work: The authors suggest exploring other popular classifiers beyond the two-layer Multilayer Perceptron (MLP) used in their experiments, such as tree-based methods.
Summary updated