Application of AHP to security protection ability comprehensive evaluation during classified evaluation

Jing Yuan, Weihong RenProceedings of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013) 2013 · DOI

Abstract

On the basis of analyzing the situation of classified evaluation, this paper brings forward a security protection ability evaluating indicators system based on the Analytic Hierarchy Process (AHP). According to the algorithm of AHP, the hierarchical architecture, the computing method of indicator weights, and synthetic weight are established. Finally, the problem of security protection ability evaluation in classified evaluation is solved.

Introduction

In the landscape of national information security, standards such as the GB/T22239-2008 Baseline provide a robust foundation of security controls. However, these standards often lead to a "forest for the trees" problem during audits. Current evaluations typically focus on binary pass/fail checklists for specific components or generalized system behavior. There is a distinct lack of a standardized, mathematical method to translate hundreds of individual checkmarks into a single, meaningful metric representing a system's overall security protection ability.

This research addresses this gap by applying the Analytic Hierarchy Process (AHP) to create a structured framework that weights security aspects based on their relative importance, moving from qualitative checklists to quantitative evaluation.

The AHP Framework for Security Evaluation

The core of the proposed method is the construction of a five-level hierarchy that maps granular security controls to high-level strategic goals.

Building the Hierarchy

The indicators are organized into the following levels:

  • Level A (Goal): Comprehensive security protection ability.
  • Level B (Ability Level): Divided into business information security (B1) and system service security (B2).
  • Level C (Ability Sublevel): Seven categories including protection, detection, response, and engineering.
  • Level D (Ability Basal Level): Twenty specific categories such as physical environment, access control, and incident management.
  • Level E (Security Controls): 264 specific controls derived from the national baseline.
The hierarchical architecture of the security protection ability evaluation system.

Methodology: From Subjectivity to Objectivity

To determine the relative importance of these indicators, the authors utilized expert questionnaires to construct judgment matrices. Through pairwise comparisons, the relative weight of each indicator is calculated using characteristic vectors.

A critical component of this process is Consistency Testing. To ensure that expert judgments are not contradictory, a Coincidence Rate (C.R.) check is performed. A C.R. of less than 0.1 indicates that the weights are mathematically sound and the model's internal logic is stable. This framework effectively forces subjective expert opinions into a consistent mathematical structure, reducing the "gut feeling" element often found in security audits.

Context-Dependent Weighting and Grades

A significant insight of this research is that security priorities are not static; they must change based on the system's required protection level. The model defines five distinct grades of security protection ability, each with corresponding management maturity and technological strength requirements.

The definition of security protection ability grades, management maturity, and technology strength.

For example, in a Grade 2 system, the evaluation might shift weight toward "effective detection," whereas a Grade 1 system focuses on basic "prevented" defense. By establishing "synthetic weights," the model can take a score from a very specific control (e.g., network access control at Level E) and mathematically propagate its impact up to the final Level A score.

Key Results and Implementation

The authors demonstrated the calculation process for a Grade 2 system, where "Business Information Security" (B1) was weighted at 0.4 and "System Service Security" (B2) at 0.6. The consistency check for Level C weights yielded a C.R. of 0.0089, well within the acceptable threshold. The final output is a formula that produces a Synthetic Score, allowing auditors to provide a quantified, defensible grade for a system's security posture.

Limitations and Future Work

While the AHP framework provides a rigorous mathematical structure, it remains dependent on the quality of the initial expert questionnaires. Biases or gaps in domain knowledge among experts can influence the final weights. Furthermore, while Levels A through C are dynamic, the weights for the lower levels (D and E) are currently treated as stable across different grades. Future research could explore making these lower-level controls more responsive to specific industry threat models or empirical breach data.

Summary updated

Abstract

On the basis of analyzing the situation of classified evaluation, this paper brings forward a security protection ability evaluating indicators system based on AHP. According to the algorithm of AHP, the hierarchical architecture, the computing method of indicator weights and synthetic weight are established. Finally, the problem of security protection ability evaluation in classified evaluation is solved.
BibTeX
@inproceedings{yuan2013application,
  author = {Jing Yuan and Weihong Ren},
  title = {{Application of AHP to security protection ability comprehensive evaluation during classified evaluation}},
  booktitle = {Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013)},
  year = {2013},
  doi = {10.2991/iccsee.2013.21},
}
Figure