The Lannion report on Big Data and Security Monitoring Research

Laurent d'Orazio, Jalil Boukhobza, Omer F. Rana, Juba Agoun, Le Gruenwald, Hervé Rannou, Elisa Bertino, Mohand-Saïd Hacid, Taofik Saïdi, Georges Bossert, Van Long Nguyen Huu, Dimitri Tombroff, Makoto Onizuka

Introduction

Modern cybersecurity is increasingly defined by the challenge of monitoring massive, continuous data streams from diverse sources such as server logs, network traffic, and IoT sensors. As infrastructures like 5G and smart cities expand, traditional security tools often fail to manage the volume, velocity, and variety of this data. This report summarizes the findings of a 2022 workshop focused on the intersection of Big Data and cybersecurity monitoring, identifying critical research bottlenecks and emerging solutions across four main pillars: specialized data processing, distributed graph analytics, privacy-aware sharing, and hardware acceleration.

Research Challenges

The core issue addressed is the effective application of Big Data technologies to security monitoring. Key sub-problems include:

  • The necessity for specialized databases to handle temporal data.
  • The complexity of scaling graph analytics without prohibitive overhead.
  • The tension between collaborative threat intelligence sharing and data privacy.
  • Legacy software bottlenecks that hinder the utilization of modern hardware like FPGAs and Non-Volatile Memory (NVM).

Methodological Innovations

Time-Series Analytics

To track sophisticated, "low and slow" intrusions, researchers are moving beyond standard relational databases toward specialized systems like Warp10. These systems utilize concatenative programming languages (e.g., WarpScript) to perform anomaly detection on Geo Time Series (GTS) data, which uniquely identifies series through classes and labels.

Structure of a Geo Time Series (GTS) object, illustrating the organization of classes, labels, and datapoints.

Graph Neural Networks (GNNs)

Graph Convolutional Networks (GCNs) are powerful for network analysis but suffer from "over-smoothing" in deep architectures, where node embeddings become indistinguishable. To mitigate this, the Adaptive Node Embedding Propagation Network (ANEPN) was proposed, which adaptively increases propagation hops to maintain distinct class boundaries for anomaly detection.

Distributed Query Optimization

The OptIQ framework introduces query optimization techniques designed to eliminate redundant computations in iterative machine learning queries. This is particularly effective for workloads running on distributed frameworks like Apache Spark and MapReduce, where it has demonstrated performance gains up to 5x.

Hardware Acceleration

To bypass traditional compute bottlenecks, the MASCARA-FPGA cooperative model offloads semantic caching and query rewriting tasks from the CPU to FPGAs. This architecture enables more efficient range query processing for massive distributed datasets by utilizing specialized semantic matching accelerators.

The MASCARA-FPGA architecture, showing the integration of semantic matching accelerators and query process controllers within the FPGA.

Privacy-Preserving Data Sharing

Collaborative security requires sharing data across organizations without violating local access control policies. Proposed methodologies involve entity-matching and policy-oriented frameworks that allow querying shared data while preventing sensitive information leakage through inference.

Key Insights

  • Specialization is Mandatory: Standard log searchers are insufficient for modern threats; optimized time-series databases are essential for long-term intrusion tracking.
  • Addressing GNN Depth: Adaptive propagation is necessary to prevent node embeddings from becoming identical in deep graph networks.
  • Software-Hardware Alignment: Legacy software stacks often negate the performance gains of emerging hardware. There is a critical need to push computation down to the storage/memory layer (in-memory computing).
  • The Need for Explanation: Real-time outlier detection must be accompanied by explanations (causality and ranking) to be actionable for administrators.

Future Directions

Future research should focus on developing algorithms for real-time outlier explanation on GPUs for streaming data. Other priorities include advancing knowledge representation, deeper AI integration in threat detection, and improving spatial-temporal data analysis to better secure evolving digital infrastructures.

Summary updated

Abstract

During the last decade, big data management has attracted increasing interest from both the industrial and academic communities. In parallel, Cyber Security has become mandatory due to various and more intensive threats. In June 2022, a group of researchers has met to reflect on their community’s impacts on current research challenges. In particular, they have considered four dimensions: (1) dedicated systems being data processing and analytic platforms or time series management systems; (2) graphs analytics and distributed computation; (3) privacy; and (4) new hardware.
BibTeX
@inproceedings{dorazio2022lannion,
  author = {Laurent d'Orazio and Jalil Boukhobza and Omer F. Rana and Juba Agoun and Le Gruenwald and Hervé Rannou and Elisa Bertino and Mohand-Saïd Hacid and Taofik Saïdi and Georges Bossert and Van Long Nguyen Huu and Dimitri Tombroff and Makoto Onizuka},
  title = {{The Lannion report on Big Data and Security Monitoring Research}},
  booktitle = {IEEE Big Data},
  year = {2022},
  doi = {10.1109/bigdata55660.2022.10020852},
}
Figure