Hardening with Scapolite: A DevOps-based Approach for Improved Authoring and Testing of Security-Configuration Guides in Large-Scale Organizations

Patrick Stöckle, Ionut Pruteanu, Bernd Grobauer, Alexander Pretschner

TL;DR

This paper introduces Scapolite, a YAML-based format and DevOps-inspired workflow for creating, managing, and testing security-configuration guides. By centralizing guide information and automating artifact generation and testing, it significantly reduces errors and improves the efficiency of security hardening in large organizations.

What Problem Does This Paper Address?

Ensuring the security of IT systems often relies on security-configuration guides that specify secure settings for operating systems and applications. However, the process of creating, tailoring, and implementing these guides is typically manual, error-prone, and time-consuming. Existing guides, often in SCAP formats, provide machine-readable checks but lack robust, integrated mechanisms for automated implementation. This leads to inconsistencies, increased risk of security vulnerabilities, and a significant burden on administrators, especially in large organizations with diverse IT infrastructures and evolving threat landscapes. The paper aims to streamline this process by introducing a more integrated and automated approach.

Methods

The core of the paper's contribution lies in its Scapolite format and its integration into a DevOps/Continuous Integration (CI) workflow.

  • Scapolite Format: This is a YAML-based format designed to be more human-readable than SCAP's XML. It centralizes all information related to a security configuration requirement: the human-readable description, machine-readable implementation details (e.g., registry keys, PowerShell commands), and validation checks. This single source of truth aims to reduce inconsistencies.
  • DevOps/CI Workflow: The authors propose a workflow where Scapolite guides are managed in version control (GitLab). Automated pipelines then:
  • Generate Artifacts: Transform the Scapolite specifications into machine-readable artifacts for implementation (e.g., PowerShell scripts for Windows) and checking.
  • Automated Testing: Spin up virtual machines (on AWS EC2), apply the generated artifacts, and run validation checks. This includes initial checks, applying settings, post-application checks, and reverting settings.
  • Test Specification: A YAML file defines the test runs, activities, and expected outcomes, also managed under version control. This allows for regression testing and comparison with previous results.
  • Transformations: The system supports transforming abstract specifications into concrete implementation and check mechanisms for various targets (e.g., Windows registry, OVAL checks).

Datasets

Not discussed in the paper. The paper focuses on a methodology and tooling rather than training or evaluating on specific datasets. The "input" to their system is external security-configuration guides (like CIS or DISA benchmarks), which are then processed and transformed.

Key Insights and Ideas

  • Centralization is Key: Consolidating all aspects of a security configuration guide (description, implementation, checks) into a single, machine-readable format like Scapolite drastically reduces the potential for errors and inconsistencies that arise from managing separate artifacts.
  • DevOps for Security Hardening: Applying DevOps principles (CI/CD, automated testing, version control) to security hardening transforms it from a manual, burdensome task into an efficient, auditable process.
  • Automated Testing Catches Errors Early: The combinatorial explosion of test cases (due to different OS versions, profiles, etc.) is a major hurdle. Automating test execution and analysis allows for frequent regression testing, catching errors introduced by guide changes or infrastructure updates much earlier.
  • YAML for Readability: While SCAP is the standard, the authors found that YAML is more approachable for guide authors, potentially increasing adoption and quality of contributed guides.
  • Abstract Specifications Reduce Maintenance Burden: By defining requirements at a higher level of abstraction, authors don't need to manage numerous low-level implementation scripts directly, simplifying updates and reducing the risk of errors.

Key Results

The paper emphasizes the benefits derived from their approach rather than specific quantitative results on a benchmark dataset.

  • Reduced Errors: The automation and centralization significantly reduce errors caused by manual transcription and inconsistent management of security configurations.
  • Increased Efficiency: The automated generation of artifacts and testing pipelines dramatically speeds up the process of authoring, maintaining, and deploying security configurations.
  • Improved Compliance: By making the process more robust and less error-prone, organizations can achieve and maintain higher levels of security compliance more effectively.
  • Early Detection of Problems: The automated testing catches the majority of problems before a security-configuration guide is released.
  • Adaptability: The approach supports tailoring guides for different use-cases and environments, and the automated testing validates these tailored versions.

Limitations and Future Work

  • Manual Effort for Initial Conversion: While Scapolite aims for automation, converting existing SCAP guides or adding machine-readable implementation details still requires manual effort, though this effort would be necessary regardless.
  • Complexity of Transformations: Developing robust transformations from abstract specifications to concrete artifacts can be complex, especially for diverse targets and platforms.
  • User Feedback Analysis: While testing is automated, the analysis of test results still requires human interaction. The paper notes the need for presenting results concisely to facilitate this analysis.
  • Future Work: The authors mention ongoing work on libraries for creating and manipulating Scapolite content, transforming abstract automations into more concrete ones, and further transforming them into code or other artifacts. They also highlight the need for continued development of the technological basis for their process.

Summary updated

Abstract

Security Hardening is the process of configuring IT systems to ensure the security of the systems' components and data they process or store. In many cases, so-called security-configuration guides are used as a basis for security hardening. These guides describe secure configuration settings for components such as operating systems and standard applications. Rigorous testing of security-configuration guides and automated mechanisms for their implementation and validation are necessary since erroneous implementations or checks of hardening guides may severely impact systems' security and functionality. At Siemens, centrally maintained security-configuration guides carry machine-readable information specifying both the implementation and validation of each required configuration step. The guides are maintained within git repositories; automated pipelines generate the artifacts for implementation and checking, e.g., PowerShell scripts for Windows, and carry out testing of these artifacts on AWS images. This paper describes our experiences with our DevOps-inspired approach for authoring, maintaining, and testing security-configuration guides. We want to share these experiences to help other organizations with their security hardening and, thus, increase their systems' security.
BibTeX
@inproceedings{stockle2022hardening,
  author = {Patrick Stöckle and Ionut Pruteanu and Bernd Grobauer and Alexander Pretschner},
  title = {{Hardening with Scapolite: A DevOps-based Approach for Improved Authoring and Testing of Security-Configuration Guides in Large-Scale Organizations}},
  booktitle = {Codaspy},
  year = {2022},
  doi = {10.1145/3508398.3511525},
}
Figure