CELEST: Federated Learning for Globally Coordinated Threat Detection

Talha Ongun, Simona Boboila, Alina Oprea, Tina Eliassi-Rad, Jason Hiser, Jack Davidson

TL;DR

CELEST is a new federated learning framework that allows organizations to collaboratively detect global cyber threats without sharing sensitive raw logs. By combining privacy-preserving model training with a novel active learning component and a robust defense against malicious participants, CELEST can identify sophisticated attacks—like data exfiltration—that are often invisible to individual networks.

What Problem Does This Paper Address?

Modern cyberattacks are increasingly coordinated and global in scale, yet our defenses remain largely siloed. While organizations often share "Indicators of Compromise" (IoCs) like malicious IP addresses, these are reactive and easily bypassed when attackers rotate their infrastructure.

The core challenge is that training powerful machine learning models requires massive amounts of data, but sharing raw security logs (like HTTP traffic) across organizations is a privacy and security nightmare. Furthermore, supervised models struggle with "zero-day" attacks—threats never seen during training. The authors aim to bridge this gap by enabling proactive, collaborative defense that respects data privacy and evolves to meet new threats.

Methods

The authors propose CELEST (CollaborativE LEarning for Scalable Threat detection), which utilizes several key innovations:

  • Federated Learning (FL): Instead of sending logs to a central server, clients (organizations) train models locally on their own data. They only share model updates (weights), which are aggregated by a central server to create a "global" model that benefits from everyone's experience.
  • Federated Feature Representation: To handle complex text fields like URLs, CELEST uses a federated version of FastText. This approach creates semantic embeddings that understand the structure of a URL rather than just looking for specific keywords. Because it uses n-grams, it can even handle new, unseen tokens at deployment time. Federated embedding model training for URL representation
  • Active Federated Learning: To tackle the lack of labeled malicious data, CELEST integrates an active learning loop. It uses both the global model's uncertainty and a local anomaly detector (Isolation Forest) to flag highly suspicious samples. These are then investigated by human analysts and used to augment the training set, allowing the system to "discover" new malware families on the fly. The Active Federated Learning framework
  • DTrust (Distributed Trust): Federated learning is vulnerable to "poisoning" attacks where a malicious client tries to corrupt the global model. DTrust allows benign clients to monitor the global model's performance on a small, verified "trust dataset." If performance drops, the server is alerted to investigate and remove the offending client's updates.
Fig. 1: Federated embedding model training for URL representation. We generate embedded features for URL, Domain and Referer. In addition to embedded features, we also include numerical features and categorical features.
Fig. 2: The Active Federated Learning framework. At time step t, the global model Gt is used to rank the unlabeled data of client i. In addition, an anomaly detector is used to rank the remaining unlabeled data samples by their anomaly score. The top ranked samples are investigated and labeled by a human expert and used in the next iterations.

Datasets

The system was evaluated using a diverse mix of real-world and synthetic data:

  • University Network Traffic: Real HTTP logs from two large university networks (averaging 29 million events per day).
  • IoT Malware: Traces from the notorious Mirai and Gafgyt malware families.
  • Data Exfiltration Malware (DEM): A dataset containing multiple families like Shakti, Pony, and SpyEye.
  • Attack Recreation Dataset (ARD): The authors set up their own vulnerable infrastructure to perform controlled, realistic attacks (including evasive maneuvers) to generate high-quality ground truth.

Key Insights and Ideas

  • The Power of Diversity: A global model trained across multiple networks captures a much wider range of malware characteristics. This allows a network to detect a threat it has never seen before simply because another participant in the federation has.
  • Bootstrapping from Zero: The active learning component is remarkably effective. Even when starting with zero labels for a specific malware family, the system can reach high detection accuracy by intelligently selecting which samples a human should review.
  • Semantic Understanding Matters: Moving beyond simple lexical features (like URL length) to deep learning embeddings significantly improves the model's ability to generalize and resist simple evasion tactics by attackers.
  • Collaborative Defense is Resilient: By leveraging the "good faith" of the majority of participants, DTrust can effectively neutralize sophisticated poisoning attacks that would otherwise render a federated model useless.

Key Results

  • Massive Performance Gains: In data exfiltration scenarios, the global model achieved a three-fold increase in Precision-Recall AUC compared to local models.
  • Rapid Learning: The global model typically outperforms local models in less than 10 iterations (about 5 hours of traffic). Federated model progress for detecting malware families
  • Real-World Discovery: During deployment on two university networks, CELEST identified 42 previously unknown malicious URLs and 20 domains in a single day, all later confirmed by VirusTotal.
  • High Precision, Low Noise: The system maintained extremely low false positive rates (lower than $3.3 \times 10^{-5}$), which is critical for preventing "alert fatigue" in security operations centers.
  • Poisoning Recovery: In tests where malicious clients dropped the model's PR-AUC from 0.93 to 0.11, the DTrust mechanism successfully identified the attackers and restored performance to 0.93.
Federated model progress for detecting malware families

Limitations and Future Work

While highly effective, the authors note a few areas for improvement:

  • Encrypted Traffic: CELEST relies on HTTP headers. As more traffic moves to encrypted HTTPS, the available feature set decreases, though the authors show the system still performs reasonably well with limited data.
  • Human-in-the-loop: The active learning component still requires expert analysts to verify flagged samples, which can be a bottleneck.
  • Centralized Aggregator: While the data is decentralized, the model aggregation still happens on a central server. Future work could explore Peer-to-Peer (P2P) architectures to remove this single point of failure.

Summary updated

Abstract

The cyber-threat landscape has evolved tremendously in recent years, with new threat variants emerging daily, and large-scale coordinated campaigns becoming more prevalent. In this study, we propose CELEST (CollaborativE LEarning for Scalable Threat detection, a federated machine learning framework for global threat detection over HTTP, which is one of the most commonly used protocols for malware dissemination and communication. CELEST leverages federated learning in order to collaboratively train a global model across multiple clients who keep their data locally, thus providing increased privacy and confidentiality assurances. Through a novel active learning component integrated with the federated learning technique, our system continuously discovers and learns the behavior of new, evolving, and globally-coordinated cyber threats. We show that CELEST is able to expose attacks that are largely invisible to individual organizations. For instance, in one challenging attack scenario with data exfiltration malware, the global model achieves a three-fold increase in Precision-Recall AUC compared to the local model. We also design a poisoning detection and mitigation method, DTrust, specifically designed for federated learning in the collaborative threat detection domain. DTrust successfully detects poisoning clients using the feedback from participating clients to investigate and remove them from the training process. We deploy CELEST on two university networks and show that it is able to detect the malicious HTTP communication with high precision and low false positive rates. Furthermore, during its deployment, CELEST detected a set of previously unknown 42 malicious URLs and 20 malicious domains in one day, which were confirmed to be malicious by VirusTotal.
BibTeX
@article{ongun2022celest,
  author = {Talha Ongun and Simona Boboila and Alina Oprea and Tina Eliassi-Rad and Jason Hiser and Jack Davidson},
  title = {{CELEST: Federated Learning for Globally Coordinated Threat Detection}},
  journal = {IEEE Trans. Inf. Forensics Secur.},
  year = {2022},
  url = {https://openreview.net/forum?id=OCpJ5_qCG5},
}
Figure