Introduction
As network speeds escalate toward 100 Gbps and beyond, the demand for intelligent data planes (IDPs) capable of real-time traffic analysis—such as malware detection and resource management—has intensified. However, existing IDP solutions face a critical "trilemma" involving flexibility, performance, and data-plane awareness. Most hardware accelerators are either too rigid to support evolving neural network (NN) architectures, introduce prohibitive latency into the packet forwarding path, or force the entire network switch to operate at a reduced clock speed, compromising overall throughput.
System Architecture
To overcome these challenges, we present Kaleidoscope, a high-performance, programmable co-processor that decouples inference from the main forwarding pipeline. Kaleidoscope utilizes a "bypass" architecture, mirroring traffic to a side-channel. This design allows the primary data plane to maintain its native high-speed clock while the co-processor operates on an optimized asynchronous clock, ensuring that standard network operations are never slowed down by complex inference tasks.
As shown in the architecture overview, Kaleidoscope includes a traffic monitor that identifies flows and routes them to the appropriate inference path. Results are written back to a query table, which the data plane can access with a negligible overhead of approximately 15.5 ns.
Heterogeneous Inference Engine
Kaleidoscope recognizes that network traffic is non-uniform, typically following a power-law distribution where a small number of "elephant flows" account for the majority of volume, while numerous "mouse flows" are short-lived. To optimize resource allocation, Kaleidoscope employs a heterogeneous inference engine:
- Fast Process Element (FPE): Optimized for "mouse flows," providing ultra-low latency inference using Matrix-Vector operations.
- Heavy Process Element (HPE): Designed for "elephant flows," utilizing a systolic array-based engine to execute complex, high-accuracy models like CNNs or RNNs.
Both elements utilize a Very-Long-Instruction-Word (VLIW) architecture, allowing them to support various NN types through firmware updates rather than hardware redesigns. Furthermore, by feeding raw packet bytes directly into the NNs, Kaleidoscope eliminates the heavy computational cost associated with traditional feature extraction.
Experimental Evaluation
We evaluated Kaleidoscope using the ISCX-VPN-2016 and USTC-TFC benchmarks for encrypted traffic analysis and malware detection. On an FPGA implementation, the system achieved a line-rate throughput of 100 Gbps, with simulations in 28nm ASIC technology reaching up to 1.6 Tbps.
Key performance metrics include:
- Latency: Inference latencies for mouse flows ranged from 256 ns to 352 ns, significantly outperforming inline accelerators like Taurus.
- Accuracy: By supporting larger models, Kaleidoscope achieved 92.8% to 99.2% accuracy, surpassing existing solutions by up to 26.5% in malware detection tasks.
- Efficiency: The base FPGA configuration requires only 7% of logic resources (LUTs), preserving capacity for other critical network functions.
Conclusion
Kaleidoscope demonstrates that decoupling inference from forwarding via a bypass architecture is a viable path toward truly intelligent, high-speed data planes. By balancing the needs of mouse and elephant flows through heterogeneous processing elements, it achieves state-of-the-art accuracy without sacrificing network performance. Future work will explore the integration of hardware sketches to support statistical feature extraction and methods to mitigate hash collisions in high-concurrency scenarios.
Summary updated