BOTracle: A Framework for Discriminating Bots and Humans

Jan Kadel, August See, Ritwik Sinha, Mathias Fischer

TL;DR

BOTracle is a multi-stage framework designed to detect sophisticated web bots in high-traffic e-commerce environments. By combining fast heuristics, semi-supervised learning on technical features, and deep graph neural networks that analyze browsing behavior, the system achieves over 98% accuracy and out-performs existing benchmarks like Botcha.

What Problem Does This Paper Address?

Bots now account for more than half of all internet traffic, driving issues ranging from price scraping and ticket scalping to election interference. The challenge is that modern bots have become "human-mimetic"—they use real browsers and legitimate user-agent strings, making them nearly indistinguishable from real users via traditional fingerprinting.

Existing detection methods often fall into two traps: they are either too slow for high-traffic sites or they rely on "static" features (like IP addresses or window sizes) that bot operators can easily spoof. Furthermore, over-reliance on intrusive challenges like CAPTCHAs degrades the user experience for humans. The authors seek a passive, multi-layered approach that prioritizes behavioral patterns, which are much harder for bots to fake without losing their operational efficiency.

Methods

The BOTracle framework uses a three-stage pipeline to balance speed and depth:

  1. Heuristics (The Filter): A fast first pass that identifies "obvious" bots. It flags forged user agents (e.g., "python-request"), perfectly regular timing between hits (suggesting automation), and unrealistic browser window sizes (e.g., axes under 50 pixels).
  2. Semi-Supervised Generative Adversarial Network (SGAN): For more ambiguous cases, the SGAN analyzes individual "hits" (requests). Unlike standard classifiers, the SGAN uses a discriminator that also functions as a classifier, allowing it to learn from both labeled and unlabeled data—a crucial advantage since "ground truth" labels are rare in web logs.
  3. Deep Graph Convolutional Neural Network (DGCNN): The most robust stage. It transforms a user's session into a Website Traversal (WT) Graph, where nodes represent sub-pages and edges represent navigational links. The DGCNN analyzes these graphs to identify non-human navigation patterns, such as exhaustive breadth-first searches or specific page-refresh frequencies.

Datasets

The researchers evaluated BOTracle using a massive, real-world dataset:

  • Source: A proprietary e-commerce platform generating $500M–$1B in annual revenue.
  • Scale: Approximately 40 million monthly page visits; the study used a subset of 1.4 million visits.
  • Ground Truth Labeling: Since real-world data lacks perfect labels, the authors used two strict assumptions:
  • Human: Traffic from known internal employee accounts.
  • Bot: Traffic originating from known cloud provider IP ranges (AWS, Azure, etc.).
  • Comparison: The framework was benchmarked against Botcha, a known bot detection framework, using the same dataset.

Key Insights and Ideas

  • The "Static Feature" Trap: The authors found that while technical features like browser_height and browser_width were the most "important" for the SGAN classifier, they are the easiest for bot developers to falsify. This highlights the necessity of moving toward behavioral analysis.
  • Behavioral Resistance: Mimicking human browsing behavior (random pauses, non-linear navigation) is computationally expensive for bot operators. If a bot is forced to act exactly like a human to avoid detection, it loses its inherent advantages: speed and tirelessness.
  • Graph Efficiency: Surprisingly, the DGCNN performed exceptionally well even on very small graphs (1–3 nodes). This is because the WT graphs aggregate multiple interactions with the same page into a single node, capturing "looping" behavior that simple sequence models might miss.
  • Multi-Stage Efficiency: By using heuristics first, the system conserves expensive GPU resources, only passing the most "human-like" bot candidates to the deep learning models.

Key Results

BOTracle demonstrated high performance across all metrics, generally surpassing the Botcha baseline:

  • SGAN: Achieved 98.95% accuracy and an AUROC of 0.9886 using technical features.
  • DGCNN: Achieved 98.45% accuracy and an AUROC of 0.9892 relying strictly on behavioral navigation.
  • Heuristic Recall: The initial heuristic layer achieved a recall of 0.9988, meaning it successfully caught the vast majority of "simple" bots before they reached the AI stages.
  • Scalability: The graph-based approach proved effective across varying session lengths, maintaining near-perfect precision and recall as the number of visited pages increased.

Limitations and Future Work

  • Proprietary Data: Because the dataset is proprietary and anonymized for business reasons, other researchers cannot directly replicate the results on the same data.
  • The "Residential Proxy" Gap: The "Bot Assumption" (Cloud IPs = Bots) is a standard research proxy, but it may not capture sophisticated bots using residential IP proxies to appear like home users.
  • Detection Latency: While the system is designed for high traffic, the authors note that future work should focus on reducing the number of user interactions required before a definitive "bot" or "human" label can be assigned.
  • Evolving Mimicry: As generative AI improves, bots may become better at simulating human-like mouse movements and typing rhythms, requiring even more nuanced behavioral features.

Summary updated

Abstract

Bots constitute a significant portion of Internet traffic and are a source of various issues across multiple domains. Modern bots often become indistinguishable from real users, as they employ similar methods to browse the web, including using real browsers. We address the challenge of bot detection in high-traffic scenarios by analyzing three distinct detection methods. The first method operates on heuristics, allowing for rapid detection. The second method utilizes, well known, technical features, such as IP address, window size, and user agent. It serves primarily for comparison with the third method. In the third method, we rely solely on browsing behavior, omitting all static features and focusing exclusively on how clients behave on a website. In contrast to related work, we evaluate our approaches using real-world e-commerce traffic data, comprising 40 million monthly page visits. We further compare our methods against another bot detection approach, Botcha, on the same dataset. Our performance metrics, including precision, recall, and AUC, reach 98 percent or higher, surpassing Botcha.
BibTeX
@inproceedings{kadel2024botracle,
  author = {Jan Kadel and August See and Ritwik Sinha and Mathias Fischer},
  title = {{BOTracle: A Framework for Discriminating Bots and Humans}},
  booktitle = {ESORICS Workshops},
  year = {2024},
  url = {https://openreview.net/forum?id=JdseO2HKXY},
}

Loading related content...

Figure