TL;DR
Bug bounty programs (BBPs) are often viewed as a monolith of "hunters" chasing rewards, but this paper reveals a deep divide in contributor profiles. By analyzing over 2,500 contributors, the researchers identified a crucial distinction between "project-specific" specialists—who act more like dedicated open-source developers—and "non-specific" hunters who move quickly between programs for financial gain.
What Problem Does This Paper Address?
As software grows more complex, companies and open-source projects increasingly rely on bug bounty programs to find security vulnerabilities. However, managing these programs is difficult: reports can dry up, and retaining high-quality talent is a constant struggle.
The authors argue that software managers lack empirical knowledge about who is actually participating in these programs. Without understanding the different motivations and work habits of contributors, managers can't effectively incentivize them or build sustainable security ecosystems. The paper seeks to answer: What contributor heterogeneity exists in bug bounty programs?
Methods
The researchers employed a multi-stage approach:
- Data Mining: They scraped data from 82 active bug bounty programs (including those on platforms like Bugcrowd and HackerOne, as well as self-hosted programs like Google’s Chromium and Mozilla).
- Identity Merging: Because contributors often use different handles across platforms, the authors manually merged identities using names, URLs, and social media links to create a clean dataset of 2,504 distinct individuals.
- Archetypal Analysis: This is a statistical technique that identifies "pure types" (archetypes) within a dataset. Unlike cluster analysis, which puts people into groups, archetypal analysis describes individuals as a mixture of these pure types.
- Surveying: To validate their statistical findings, they conducted a quantitative and qualitative survey with active contributors from different archetypes.
Datasets
- Programs: 82 active BBPs across various domains (Browsers/OS, Web/Mobile, Digital Currency, and Programming Languages).
- Contributors: 2,504 distinct individuals.
- Bounty Data: Ranges from small $25 rewards to $100,000 critical vulnerability payouts.
Key Insights and Ideas
- The 80/20 Rule (and then some): While 60% of contributors only ever received one bounty, the top 20% of contributors were responsible for 64% of all successful reports.
- Three Distinct Archetypes:
- A1 (The Casual): Less active, usually rewarded only a few times, often on high-profile web targets like Yahoo or Facebook.
- A2 (The Specialist): Highly focused on specific, complex projects (e.g., Chromium or Mozilla). They spend significant time (days to weeks) on a single bug.
- A3 (The Hunter): Active across many different programs. They are highly efficient, typically spending less than half a day to find and report a bug.
- Motivation Mismatch: Specialists (A2) see themselves as "security researchers" or "developers" and are motivated by product security and reputation. Hunters (A3) see themselves as "bug bounty hunters" and are primarily driven by the bounty amount.
- The "Peer Production" Mindset: Project-specific contributors (A2) behave more like traditional open-source contributors. They use the products they test and want a two-way "information flow" with the vendor, rather than just a transactional relationship.
Key Results
- Time Investment: There is a massive gap in effort. A3 contributors (Hunters) reported spending at most half a day on a bug, while A2 contributors (Specialists) often worked for a full week.
- Market Risks: One of the most striking findings was that A2 specialists—because of their high time investment—are more likely to consider selling bugs to "black/gray markets" if the official bounty doesn't feel commensurate with their effort.
- Retention: The study found that attracting new contributors is vital, as 36% of total bounties were paid to the "less active" majority.
Limitations and Future Work
The primary limitation is the small survey sample size; while the data mining covered thousands, only seven contributors responded to the deep-dive survey. Additionally, the manual identity merging process, while more accurate than automated scripts, is subject to human error and is difficult to scale.
For future work, the authors suggest looking "beyond the onion model" of open-source development. Traditional models suggest contributors move from the outside in (user -> reporter -> developer), but bug bounty contributors seem to occupy a unique, permanent role outside the standard hierarchy that requires its own management framework.
Summary updated