TL;DR
This paper tackles the "black box" problem of deep learning intrusion detection systems (IDS) in industrial control networks (ICN). They propose a novel method combining a modified Layer-wise Relevance Propagation (LRP) with a new data normalization technique to explain why an intrusion is detected, mapping network anomalies to specific attributes. This allows security professionals to more quickly and accurately identify and address threats in critical infrastructure.
What Problem Does This Paper Address?
Industrial control systems (ICS) are critical for infrastructure like power grids and manufacturing. Intrusion detection is vital, but current deep learning-based IDS, while powerful, often act as "black boxes." They raise alarms but provide little insight into what specific network behaviors triggered the alert. This lack of interpretability forces human analysts to perform extensive, time-consuming investigations to understand the nature of an attack, which is problematic in time-sensitive ICS environments. The paper aims to bridge this gap by making deep learning IDS in ICNs more explainable, providing actionable insights beyond a simple alarm.
Methods
The core of the proposed solution involves two main innovations:
- Novel Data Normalization: Traditional min-max scaling can compress normal data into a very small range, hindering anomaly detection. The authors introduce a generalized normalization method that splits data into two dimensions to better preserve the discrimination of normal and abnormal data, especially for industrial control network traffic which exhibits strong regularities.
- Layer-wise Relevance Propagation (LRP) for Attribute Explanation: Inspired by LRP, the authors adapt it to map the "abnormalities" found during the DNN's calculation process back to the specific input attributes. This involves identifying a "Key Decision Layer" (KDL) within the DNN where the most significant decision-making occurs for a given anomaly. By comparing the calculation process of abnormal samples with normal ones, they pinpoint deviations and then use a modified LRP to trace these deviations back to the input features, effectively explaining which attributes contributed to the detection.
Datasets
The primary dataset used is a gas pipeline dataset from the Industrial Control System Traffic Datasets for Intrusion Detection Research (Morris and Gao, 2014). This dataset captures MODBUS traffic from an RS-232 connection and includes normal data along with seven categories of abnormal data representing various intrusion types (NMRI, CMRI, MSCI, MPCI, MFCI, DoS, Reconnaissance). The paper also mentions using an eight-class dataset for relevance calculation to verify applicability.
Key Insights and Ideas
- Deep Learning's "Black Box" is a Bottleneck for ICS Security: The paper strongly emphasizes that while deep learning excels at detection, its lack of interpretability is a major hurdle for critical infrastructure security where human expertise is paramount.
- Normal ICS Traffic Has Predictable Regularities: The authors leverage the inherent structured nature of normal ICS traffic to design a more effective normalization method.
- Key Decision Layers (KDLs) Hold the Key: Identifying specific hidden layers within the DNN that are most crucial for making classification decisions is central to their explanation method.
- Mapping Calculation Anomalies to Input Attributes: The core idea is that deviations in the DNN's internal calculations during an attack can be traced back to specific input features, providing a direct link between network behavior and the detection.
- Reduced Training Data Cost: The proposed normalization method and the focus on explanation can potentially reduce the cost of collecting and labeling extensive training datasets.
Key Results
- Improved Normalization: The new normalization method showed improvements in training accuracy, particularly at lower learning rates, making it easier to achieve high accuracy (e.g., >95% with dynamic learning rates).
- Hidden Layer Analysis: Experiments demonstrated that clustering outputs from hidden layers could effectively differentiate between various attack types, supporting the idea that these layers capture distinct "stepwise-bases" of attacks.
- Relevance Calculation: The LRP-based method successfully identified attributes contributing to detected anomalies. For example, it highlighted specific parameters like "measurement" and "resp_read_fun" for certain attack types, helping analysts focus their investigation.
- Attack Type Identification: The method showed promise in narrowing down the scope of investigation, even when the specific attack class was not fully known beforehand. For instance, it could distinguish between command injection and response injection attacks.
Limitations and Future Work
- Partial Derivative Accuracy: The paper acknowledges that the partial derivative calculation for nonlinear activation functions is not perfectly accurate, and negative values in the results are difficult to interpret.
- Finding Intrusion Factors: The current method cannot precisely pinpoint the "intrusion factor" but rather narrows down the scope of analysis.
- Model Complexity: The accuracy of the method might be affected by the complexity of the DNN model and the specific activation functions used (e.g., CNNs are noted as more complicated).
- Future Work: The authors suggest further research into improving the accuracy of partial derivative calculations, better explaining negative relevance values, and more precisely identifying intrusion factors. They also mention the need to adapt the method for different network structures and activation functions.
Summary updated