Fixing Outside the Box: Uncovering Tactics for Open-Source Security Issue Management

Lyuye Zhang, Jiahui Wu, Chengwei Liu, Kaixuan Li, Xiaoyu Sun, Lida Zhao, Chong Wang, Yang Liu

Introduction

In the world of Open-Source Software (OSS), identifying a vulnerability is only half the battle; the "last mile" is remediation. While current academic research and industrial tools—such as Software Composition Analysis (SCA) or Automatic Vulnerability Repair (AVR)—primarily focus on "inside the box" solutions like code patches or dependency version bumps, the reality on the ground is far more complex.

This paper explores how the OSS community manages security issues using a diverse array of "outside the box" tactics. By analyzing over 21,000 GitHub issues, the authors uncover 44 distinct remediation tactics, revealing a significant gap: 54% of CVEs lack official fix suggestions, yet the community has already solved 93% of these cases through alternative methods.

Methodology and Data

The researchers conducted a multi-stage empirical analysis to bridge the gap between theoretical fixes and real-world management.

Overview of the study methodology and data preparation process.

The study utilized a massive dataset, which the authors claim is the largest-ever collection of security issues in academia:

  • Primary Dataset: 21,187 closed security-related issues from 2,600 highly-starred GitHub repositories.
  • CVE Subset: 3,044 issues explicitly linked to CVE IDs.
  • Comparison Sources: Data from 9 major vulnerability databases, including NVD, GitHub Advisory, and Snyk.

To categorize these issues, the team employed Hybrid Card Sorting. Four experts manually labeled and grouped the issues to create a hierarchical taxonomy of 44 distinct remediation operations.

A Taxonomy of Remediation Tactics

The core contribution of this work is the identification of six major categories of Remediation Tactics (RT): Version Adjustment, Patching, Using Alternative Libraries, Changing Configurations, Bypassing, and Re-initializing.

Sankey diagram illustrating the hierarchical taxonomy of 44 remediation tactics and their distribution across the dataset.

The analysis reveals that nearly half (44%) of these community-driven tactics are currently unsupported by state-of-the-art (SOTA) tools. This "invisible" spectrum of fixes includes swapping to alternative libraries or tweaking configurations to mitigate risks without necessarily changing the core code.

Key Insights and Results

The Identity Gap: Users vs. Maintainers

The study highlights a notable difference in priorities between "Users" (downstream developers) and "Maintainers" (upstream owners). Users often prefer simple, lightweight tactics—such as installing specific components—to achieve a quick fix. In contrast, maintainers are more conservative, prioritizing long-term stability and avoiding "breaking changes" or maintenance debt. While patching is the "gold standard" for acceptance, maintainers frequently prefer "Version Adjustment" due to its predictability.

The Database Silence

A staggering 54% of CVEs in modern databases provide no remediation suggestions, creating a dangerous "attack window." However, the researchers found that for 93% of these "silent" CVEs, the GitHub community had already discussed and implemented actionable solutions. By leveraging these community discussions, the number of available patches in vulnerability databases could be increased by 7.7x.

The Cost of Complexity

Remediation is not always straightforward. The median discussion duration for a security issue is 6.1 days, but 28% of issues remain open for more than a month, giving attackers a significant head start. Furthermore, tactics like "Re-initializing" or "Changing Configurations" are often "volatile"—93% of reopened issues in these categories were due to incomplete previous fixes, indicating they are harder to implement correctly than standard patches.

Conclusion and Future Work

While this study provides a comprehensive look at OSS security management, it is limited by its manual labeling process and its focus on GitHub. Future research should aim to automate the synthesis of these tactics using Large Language Models (LLMs) or Retrieval-Augmented Generation (RAG). Standardizing these remediation tactics into formats similar to Software Bills of Materials (SBOMs) could also help security tools communicate a wider range of fix options to developers, ultimately closing the gap between vulnerability discovery and resolution.

Summary updated

BibTeX
@article{zhang2025fixing,
  author = {Lyuye Zhang and Jiahui Wu and Chengwei Liu and Kaixuan Li and Xiaoyu Sun and Lida Zhao and Chong Wang and Yang Liu},
  title = {{Fixing Outside the Box: Uncovering Tactics for Open-Source Security Issue Management}},
  journal = {Proc. ACM Softw. Eng.},
  year = {2025},
  doi = {10.1145/3728977},
}
Figure