Abstract
As advances in Deep Neural Networks (DNNs) demonstrate unprecedented levels of performance in many critical applications, their vulnerability to attacks is still an open question. We consider evasion attacks at testing time against Deep Learning in constrained environments, in which dependencies between features need to be satisfied. These situations may arise naturally in tabular data or may be the result of feature engineering in specific application domains, such as threat detection in cyber security. We propose a general iterative gradient-based framework called FENCE for crafting evasion attacks that take into consideration the specifics of constrained domains and application requirements. We apply it against Feed-Forward Neural Networks trained for two cyber security applications: network traffic botnet classification and malicious domain classification, to generate feasible adversarial examples. We extensively evaluate the success rate and performance of our attacks, compare their improvement over several baselines, and analyze factors that impact the attack success rate, including the optimization objective and the data imbalance. We show that with minimal effort (e.g., generating 12 additional network connections), an attacker can change the model's prediction from the Malicious class to Benign and evade the classifier. We show that models trained on datasets with higher imbalance are more vulnerable to our FENCE attacks. Finally, we demonstrate the potential of performing adversarial training in constrained domains to increase the model resilience against these evasion attacks.
Introduction
Most research into adversarial machine learning focuses on continuous domains like computer vision, where an attacker can modify individual pixels independently. However, in critical fields like cybersecurity, finance, or healthcare, data is typically tabular and highly constrained. Features in these domains are rarely independent; for instance, in a network log, increasing the "Total Bytes Sent" must logically reflect in the "Average Bytes per Packet."
Standard adversarial attacks, such as Carlini-Wagner or PGD, often produce "infeasible" data points that violate these logical dependencies. Such examples are easily caught by simple sanity checks or fail to execute in real-world systems. This paper introduces a framework to generate adversarial perturbations that are both effective and logically consistent within constrained environments.
The FENCE Framework
To address the challenge of feature dependency, we propose FENCE (Feasible Evasion Attacks on Neural Networks in Constrained Environments). FENCE is an iterative, gradient-based framework designed to maintain data integrity through several key mechanisms:
- Feature Families: Dependent features are grouped into families. If one feature is modified, the entire family is updated to maintain consistency.
- Iterative Optimization: The algorithm identifies the feature with the highest gradient—the one most influential to the model's prediction—at each step.
- Consistent Updates: Specialized functions update entire feature families based on domain-specific mathematical rules (e.g., ensuring ratios remain correct).
- Projection and Penalty: The framework utilizes modified versions of Projected Gradient Descent and penalty-based optimization to ensure the attack remains within the "feasibility region."
Experimental Evaluation
FENCE was evaluated on two cybersecurity tasks:
- CTU-13 Dataset: A public botnet traffic dataset where the goal was to evade detection of the "Neris" botnet.
- MADE Dataset: A proprietary enterprise dataset for malicious domain classification, characterized by extreme class imbalance (1:50 malicious to benign).
Key Results
- Success Rates: FENCE achieved a 100% success rate on the CTU-13 dataset with minimal perturbation and a 98.3% success rate on the malicious domain task.
- Stealth: On network traffic tasks, flipping a "Malicious" label to "Benign" required as few as 12 realistic network connections. These changes were statistically insignificant, making them nearly impossible to detect via standard anomaly detection.
- Transferability: Attacks generated for one neural network architecture often successfully tricked different architectures or even different model types like Random Forests.
Discussion: Constraints and Vulnerabilities
A critical finding is that standard attacks fail in tabular data because they ignore feature dependencies. As shown in the comparison below, traditional methods produce mathematically inconsistent values (e.g., an average that does not match the total and count), whereas FENCE ensures the "Adversarial Example" aligns with the "Correct Value" logic.
Furthermore, we observed that models trained on highly imbalanced datasets—common in security—are significantly more vulnerable. The inherent bias toward the majority "Benign" class makes it easier for an attacker to push a malicious sample across the decision boundary. To mitigate this, we demonstrated that adversarial training (incorporating FENCE-generated samples during training) can reduce attack success rates from nearly 100% to 16.5%.
Conclusion
FENCE demonstrates that even highly constrained tabular models are vulnerable to stealthy evasion when domain-specific dependencies are accounted for. While this work focuses on white-box Feed-Forward Neural Networks, future research will explore black-box versions and extend these constraints to sequence-based models like RNNs and Transformers in domains such as healthcare and finance.
Summary updated