VulnEx: Exploring Open-Source Software Vulnerabilities in Large Development Organizations to Understand Risk Exposure

Frederik L. Dennig, Eren Cakmak, Henrik Plate, Daniel A. Keim

Abstract

Modern software development is heavily reliant on open-source software (OSS) dependencies, which significantly expands an organization's attack surface. Managing these vulnerabilities across thousands of repositories is a major challenge for security analysts. This paper introduces VulnEx (Vulnerability Explorer), a visual analytics tool designed to provide an organization-wide overview of security risks. By shifting focus from individual applications to the entire portfolio, VulnEx enables analysts to identify widespread vulnerable libraries and prioritize remediation efforts effectively.

Introduction

The reliance on open-source libraries speeds up development but introduces systemic risks. A single vulnerability in a popular library can compromise hundreds of applications within a large organization. Current security workflows often rely on text-heavy reports focused on individual projects, leaving a gap in organizational-level visibility. Security analysts need to answer critical questions: Which projects are most at risk? Which library causes the most trouble across the portfolio? Is a specific high-profile CVE present in any system? VulnEx addresses these needs through a unified visual interface.

The VulnEx Visual Analytics Tool

VulnEx utilizes a "tree+table" visualization approach to bridge the gap between high-level overviews and technical details. Developed through a user-centered design process with security experts at SAP, the tool offers three complementary perspectives:

  1. Repository-centered: Focuses on applications and their specific risks.
  2. Library-centered: Focuses on dependencies and their prevalence across the organization.
  3. Bug-centered: Focuses on specific CVEs to track their footprint.
The VulnEx interface showing the tree-table visualization with CVSS heatmaps and CVE matrix.

Visual Encodings and Metadata

The tool employs heatmaps to represent the distribution of CVSS (Common Vulnerability Scoring System) scores, allowing for quick identification of "hotspots." A CVE Matrix shows the co-occurrence of specific bugs across different entities.

Beyond security scores, VulnEx integrates "health" metrics from GitHub (stars, watchers, issues) and LGTM (code quality grades). This metadata helps analysts assess whether a library is well-maintained or a "dead" project that should be replaced, reflecting a shift in thinking where library health is a component of security.

Integration of external health metrics like LGTM grades and GitHub stars to assess library maintenance.

Evaluation and Key Insights

The authors evaluated VulnEx using a dataset of 295 public GitHub repositories from the Eclipse Foundation, focusing on Apache Maven projects written in Java. Vulnerabilities were identified using Eclipse Steady.

The Power of Aggregation

The evaluation demonstrated that vulnerability management is an optimization problem. For instance, identifying that a single library like activemq-all affected 20 different repositories showed that one patch could significantly lower the organization's overall risk profile.

Preference for Categorical Data

Security experts preferred categorical heatmaps (Low, Medium, High, Critical) over precise numerical CVSS values. For high-level auditing, the qualitative severity ("how bad is it?") proved more actionable than minor numerical differences (e.g., 9.2 vs 9.3).

Legacy Risk Identification

In the Eclipse dataset, VulnEx quickly identified that the low-marmoset repository was exposed to multiple critical and high-severity vulnerabilities. It also helped find "ancient" bugs, such as a medium-severity vulnerability from 2009 (CVE-2009-2625) still present in 27 repositories.

Conclusion and Future Work

VulnEx provides a much-needed "big picture" for security auditing and governance. While effective for hundreds of repositories, future work will address scalability for organizations with over 1,000 repositories. The authors also plan to incorporate temporal analysis to track risk exposure over time and expand language support beyond Java and Python to ecosystems like npm. Collaborative features, such as annotating false positives, are also under consideration to enhance team workflows.

Summary updated

Abstract

The prevalent usage of open-source software (OSS) has led to an increased interest in resolving potential third-party security risks by fixing common vulnerabilities and exposures (CVEs). However, even with automated code analysis tools in place, security analysts often lack the means to obtain an overview of vulnerable OSS reuse in large software organizations. In this design study, we propose VulnEx (Vulnerability Explorer), a tool to audit entire software development organizations. We introduce three complementary table-based representations to identify and assess vulnerability exposures due to OSS, which we designed in collaboration with security analysts. The presented tool allows examining problematic projects and applications (repositories), third-party libraries, and vulnerabilities across a software organization. We show the applicability of our tool through a use case and preliminary expert feedback.
BibTeX
@inproceedings{dennig2021vulnex,
  author = {Frederik L. Dennig and Eren Cakmak and Henrik Plate and Daniel A. Keim},
  title = {{VulnEx: Exploring Open-Source Software Vulnerabilities in Large Development Organizations to Understand Risk Exposure}},
  booktitle = {VizSec},
  year = {2021},
  doi = {10.1109/vizsec53666.2021.00014},
}
Figure