Abstract
Modern cybersecurity often faces a trade-off between isolated, ineffective monitoring and privacy-invasive data sharing. This paper presents a novel approach that integrates network sensing directly into the operating system layer. By leveraging the Database Operating System (DBOS) and GraphBLAS, we demonstrate a "collaborative awareness" framework. This system represents network traffic as hypersparse matrices, allowing distributed services to share anonymized patterns to detect coordinated threats. Benchmarks on the MIT SuperCloud show the system sustains over 100,000 requests per second with linear scalability, proving that high-performance security can be an intrinsic feature of cloud infrastructure.
Introduction
Traditional security architectures typically rely on Intrusion Detection Systems (IDSs) that operate in "stovepipes"—isolated silos that monitor only local traffic. While effective for local anomalies, these silos fail to detect sophisticated, coordinated attacks spanning multiple services. Conversely, "radical transparency," where organizations share raw logs, is often precluded by privacy regulations and data volume.
We propose Collaborative Awareness as a middle ground. The objective is to build a system capable of aggregating and analyzing network traffic across distributed nodes in real-time, maintaining minimal computational overhead while preserving user privacy.
Methodology: DBOS and GraphBLAS
The core of our approach lies in the mathematical representation of network traffic. Instead of traditional text-based logging, we utilize hypersparse matrices via the GraphBLAS standard.
Hypersparse Representation
Network traffic is represented as source/destination IP pairs within a matrix of $2^{32} \times 2^{32}$ dimensions. Despite the massive theoretical size, the actual traffic is "hypersparse," allowing GraphBLAS to perform high-speed linear algebra operations to calculate metrics like unique links or fan-out with extreme efficiency.
System Integration
We implemented this sensing within the Database Operating System (DBOS). In DBOS, system services are transactional procedures on a high-performance database. This allows monitoring to be embedded directly into the "runtime fabric" of the OS. We explored two implementation paths: a high-throughput Python-GraphBLAS approach (using pPython) and a durable OneSparse PostgreSQL extension.
Hierarchical Aggregation
As shown in the workflow, local worker nodes aggregate traffic into base matrices. These are periodically merged into a locally aggregated matrix, which is then sent to a coordinator node to form a global view of the network. This hierarchy ensures that only structural patterns—rather than raw user data—are shared.
Performance Evaluation
To evaluate the system, we used synthetic traffic based on the IEEE/MIT/Amazon Anonymized Network Sensing Graph Challenge, targeting a constant injection rate of $10^5$ requests per second (RPS). Testing was conducted on the MIT SuperCloud using up to 64 compute nodes.
Key Results
- Throughput: A single DBOS instance successfully sustained over $10^5$ RPS, exceeding the requirements of most real-world web deployments.
- Scalability: The Python-based implementation exhibited linear scalability up to 64 nodes. The PostgreSQL version scaled linearly up to 32 nodes.
- Latency: Global analysis was completed in approximately 1.9 seconds for the Python workflow and 12.9 seconds for the PostgreSQL workflow.
- Overhead: The computational cost of maintaining this level of awareness was found to be negligible relative to the total system resources.
Conclusion and Future Work
By embedding sensing into the OS/Database layer, developers can achieve "security as a side effect." The use of hypersparse matrices provides a privacy-preserving mechanism to share attack "shapes" (e.g., DDoS patterns) without exposing sensitive IP addresses.
Future work will focus on addressing single-threaded bottlenecks in worker nodes, further optimizing the PostgreSQL implementation, and integrating formal prefix-preserving anonymization (e.g., CryptoPAN) into the DBOS architecture. Finally, validating the system against heterogeneous, real-world traffic remains a priority for production readiness.
Summary updated