Abstract
Nowadays, there is a fundamental imbalance between attackers and defenders. Crowdsourced tests level the playing field. However, the concern about vulnerability leaks severely limits the widespread of crowdsourced tests. Existing crowdsourced test platforms have adopt various technical or management approaches to protect applications or systems under test, but none of them is able to remove the concerns about vulnerability leaks. This paper provides a generic architecture to discover the white hat who finds a vulnerability but conceals it. The architecture is not only valid for public vulnerabilities, but also valid for unknown vulnerabilities. Finally, the proposed architecture is tested by real vulnerabilities. The results show that, with proper rules, most of the concealing behaviors can be detected.
Introduction
Crowdsourced security testing, commonly known as bug bounty programs, has emerged as a critical strategy for identifying vulnerabilities in complex systems. By leveraging a global pool of "white hat" researchers, organizations can find flaws that internal teams might miss. However, this model introduces a significant trust gap: the risk of vulnerability leaks. A researcher might discover a critical zero-day exploit and choose to conceal it for personal gain or sale on the secondary market rather than disclosing it to the platform. Current safeguards, such as Non-Disclosure Agreements (NDAs), lack technical enforcement. This paper proposes a generic monitoring architecture designed to provide real-time technical proof of such concealment behaviors.
Proposed Architecture
Network Topology and Mandatory Access Control
The core of the proposed detection strategy is the establishment of a mandatory network chokepoint. To interact with the Application or System Under Test (A/SUT), testers are required to connect via a platform-controlled VPN. A firewall is configured to drop any traffic to the A/SUT that does not originate from the VPN's IP range, ensuring total visibility into researcher activities.
As shown in the topology, a switch mirror port sends a copy of all tester traffic to a dedicated monitor server. This setup transforms the crowdsourced environment from an unmonitored "honor system" into a controlled laboratory where every packet is subject to analysis.
Hybrid Detection Engine
The software architecture is divided into three functional layers: Traffic Collection, Vulnerability Monitoring, and Result Display. The monitoring layer is the system's "brain," utilizing a hybrid approach to identify both known and unknown exploits.
- Rule-Matching Engine (Snort): This engine utilizes a database of known attack patterns (provided by EVERSEC) to identify when a tester is exploiting a known vulnerability. It provides high-precision detection for established CVEs.
- Deep Learning Engine (RNN/TensorFlow): To address the challenge of zero-day or unknown vulnerabilities, the system employs a Recurrent Neural Network (RNN). The RNN is trained on "normal" traffic baselines to recognize standard application behavior. If a tester's traffic is flagged as anomalous by the RNN but does not match any existing Snort rules, the system identifies it as a potential new discovery that the researcher may be attempting to hide.
Evaluation and Results
The architecture was evaluated using real-world vulnerabilities on the ICS-CERT crowdsourced platform. Key test cases included:
- SQL Injection (CVE-2017-8917): The system successfully detected three valid Proof of Concepts (POCs) targeting a Joomla vulnerability. It also correctly identified a fourth invalid POC, demonstrating a low false-positive rate for rule-based detection.
- XSS and ShellShock: The monitor successfully flagged reflective Cross-Site Scripting (XSS) attacks and ShellShock exploitations in real-time.
- Unknown Vulnerabilities: Preliminary tests indicated that the RNN model could successfully flag anomalous traffic patterns that bypassed traditional signature-based rules, effectively identifying the discovery of novel flaws.
Discussion and Future Work
While the VPN-based architecture provides unprecedented visibility, challenges remain regarding encrypted traffic (HTTPS). Future iterations may require SSL/TLS inspection to maintain deep packet visibility. Additionally, while the RNN approach is promising for zero-day detection, further research is needed to minimize false positives in high-noise production environments and to counter adversarial evasion tactics where testers attempt to blend their exploits into legitimate traffic streams.
Conclusion
By combining mandatory access control with a hybrid detection engine, this architecture provides a technical solution to the problem of vulnerability concealment in crowdsourced testing. This approach moves beyond legal deterrents, offering platforms a way to verify that white hat researchers are acting in good faith and disclosing all discovered flaws.
Summary updated