Understanding the Heterogeneity of Contributors in Bug Bounty Programs

Hideaki Hata, Mingyu Guo, Muhammad Ali Babar

Abstract

While bug bounty programs (BBPs) are not new in software development, an increasing number of companies and open-source projects rely on external parties to perform security assessments for rewards. However, there is relatively little empirical knowledge about the characteristics of these contributors. This paper aims to understand BBP contributors by highlighting the heterogeneity among them. We analyzed the histories of 82 programs and 2,504 distinct contributors, supplemented by a quantitative and qualitative survey. Our findings identify project-specific and non-specific contributors with distinct motivations and work habits. These insights can help organizations better manage BBPs and understand emerging software development roles.

Introduction

As software complexity increases, bug bounty programs have become a critical tool for identifying vulnerabilities. Despite their popularity, managing these programs remains a challenge for software managers who often lack empirical data on the individuals participating. Without understanding the different motivations and work habits of contributors, organizations struggle to effectively incentivize talent or build sustainable security ecosystems. This study addresses this gap by asking: What contributor heterogeneity exists in bug bounty programs?

Methodology

The researchers employed a multi-stage approach beginning with data mining from 82 active BBPs across platforms like Bugcrowd and HackerOne, as well as self-hosted programs like Google’s Chromium. After manually merging 2,504 distinct identities, they performed Archetypal Analysis to identify "pure types" within the dataset. Unlike standard clustering, this statistical technique describes individuals as a mixture of these archetypes. Finally, a survey was conducted to validate these statistical profiles with qualitative insights.

Archetypal analysis plot showing the distribution of 2,504 contributors among the three identified profiles: A1 (less active), A2 (project-specific), and A3 (non-specific).

Key Insights: The Three Archetypes

The study revealed that the top 20% of contributors are responsible for 64% of all successful reports, while 60% of participants only ever received a single bounty. From this data, three distinct archetypes emerged:

  • A1 (The Casual): The majority of the crowd. They are less active and typically rewarded only a few times, often on high-profile web targets.
  • A2 (The Specialist): Highly focused on specific, complex projects like Chromium or Mozilla. They spend significant time (days to weeks) on a single bug and act more like traditional open-source developers.
  • A3 (The Hunter): Active across many different programs. They are highly efficient, typically spending less than half a day to find and report a bug, moving quickly between targets for financial gain.

Comparative Analysis: Specialists vs. Hunters

The survey data highlighted a significant "motivation mismatch" between specialists and hunters. Specialists (A2) often identify as security researchers or developers and are motivated by product security and reputation. In contrast, Hunters (A3) see themselves primarily as bug bounty hunters driven by the bounty amount.

Survey results comparing the profiles, motivations, and time investment of project-specific specialists (A2) and non-specific hunters (A3).

One of the most striking findings concerns market risks. Because A2 specialists invest significant time (often a full week) into a single report, they are more likely to consider selling bugs to black or gray markets if the official bounty does not feel commensurate with their effort. This suggests that for complex projects, a purely transactional relationship may not be sufficient for long-term security.

Conclusion

Bug bounty contributors are not a monolith. While "Hunters" provide breadth and efficiency, "Specialists" provide depth and a peer-production mindset similar to open-source developers. Attracting and retaining both is vital: 36% of total bounties are paid to the less active majority, but the high-effort specialists are essential for securing complex systems. Future management frameworks must look beyond the standard hierarchy to effectively integrate these unique roles into the software development lifecycle.

Summary updated

Abstract

Background: While bug bounty programs are not new in software development, an increasing number of companies, as well as open source projects, rely on external parties to perform the security assessment of their software for reward. However, there is relatively little empirical knowledge about the characteristics of bug bounty program contributors. Aim: This paper aims to understand those contributors by highlighting the heterogeneity among them. Method: We analyzed the histories of 82 bug bounty programs and 2,504 distinct bug bounty contributors, and conducted a quantitative and qualitative survey. Results: We found that there are project-specific and non-specific contributors who have different motivations for contributing to the products and organizations. Conclusions: Our findings provide insights to make bug bounty programs better and for further studies of new software development roles.
BibTeX
@inproceedings{hata2017understanding,
  author = {Hideaki Hata and Mingyu Guo and Muhammad Ali Babar},
  title = {{Understanding the Heterogeneity of Contributors in Bug Bounty Programs}},
  booktitle = {ESEM},
  year = {2017},
  url = {https://openreview.net/forum?id=OFDWXsTPCHK},
}
Figure