Salient Store: Enabling Smart Storage for Continuous Learning Edge Servers

Cyan Subhra Mishra, Deeksha Chaudhary, Mahmut Taylan Kandemir, Chita R. Das

Introduction

Edge servers are increasingly evolving from simple inference endpoints into continuous learning hubs that retrain models locally to combat data drift. However, this evolution brings a massive data management burden. A single 1080p camera at 60fps can generate over 117 TiB of raw data daily. Archiving this data requires a "triple threat" of processing: compression, encryption, and redundancy (RAID).

On traditional edge hardware, these archival tasks consume significant CPU and DRAM resources, often starving the primary AI models of the compute they need to function. As shown in the table below, standard algorithms for encryption and compression impose a heavy toll on system resources.

Table 1: CPU and DRAM utilization for various data processing tasks on a 16-core Xeon processor.

Furthermore, the transition to a post-quantum security landscape necessitates computationally expensive lattice-based cryptography (LBC) to prevent "store now, decrypt later" attacks, further exacerbating the resource bottleneck.

System Architecture and Methods

Salient Store addresses these challenges through a hardware-software co-design that offloads heavy archival tasks to Computational Storage Devices (CSDs)—essentially SSDs integrated with on-board FPGAs. This shifts the architecture from a CPU-centric model to a data-centric one where storage acts as an active participant in the machine learning pipeline.

Figure 1: The Salient Store system architecture, illustrating the integration of continuous learning compute with FPGA-accelerated storage.

The system architecture relies on two primary pillars:

  1. Layered Neural Compression: Instead of using standard codecs like H.264, Salient Store reuses the feature vectors already generated by the AI inference pipeline (e.g., from a MobileNet backbone). By treating these features as a "latent space" for a neural autoencoder, the system effectively compresses video by only storing the "deltas" or residuals that the model couldn't predict.
  2. Accelerated Lattice-Based Cryptography (LBC): To future-proof data security, the authors implemented a quantum-safe encryption scheme on the CSD's FPGA. Custom hardware modules—High-Speed Schoolbook Polynomial Multiplication (HSPM) and Signed Double Modular Multiplication (SDMM)—handle the complex polynomial math required for LBC without slowing down the data flow.

Evaluation and Key Results

The researchers evaluated Salient Store using high-volume, real-world datasets relevant to autonomous driving and urban mobility, including the Kitti Vision Benchmark, nuScenes, Cityscapes, and the Waymo Open Dataset. The system's versatility was also tested with the Chime audio dataset.

Key performance metrics include:

  • Efficiency Gains: Salient Store reduced communication latency by approximately 6.2× and decreased data volume by approximately 6.1× compared to classical storage baselines.
  • Encryption Performance: The FPGA-accelerated lattice encryption provided a 3.2× speedup over its software counterpart and was 2.5× faster than software-based RSA.
  • System Throughput: In a multi-node setup, the system achieved a 4.77× speedup against traditional video storage systems (VSS).
  • Visual Quality: Despite high compression ratios, the neural codec maintained high video quality (measured via PSNR), outperforming H.264 and HEVC at various bitrates.

Discussion and Future Work

The core insight of Salient Store is the reuse of compute: the work done for AI inference is leveraged for storage efficiency rather than being discarded. By utilizing PCIe peer-to-peer communication, CSDs can communicate with each other and the main accelerator without constantly interrupting the host CPU.

While the performance gains are significant, the authors acknowledge limitations in hardware cost and power consumption. CSDs are currently significantly more expensive than standard SSDs or HDDs, and their FPGAs draw additional power. Future research will focus on optimizing these power demands and addressing network contention challenges in large-scale distributed data management protocols.

Summary updated

Abstract

As continuous learning based video analytics continue to evolve, the role of efficient edge servers in efficiently managing vast and dynamic datasets is becoming increasingly crucial. Unlike their compute architecture, storage and archival system for these edge servers has often been under-emphasized. This is unfortunate as they contribute significantly to the data management and data movement, especially in a emerging complute landscape where date storage and data protection has become one of the key concerns. To mitigate this, we propose Salient Store that specifically focuses on the integration of Computational Storage Devices (CSDs) into edge servers to enhance data processing and management, particularly in continuous learning scenarios, prevalent in fields such as autonomous driving and urban mobility. Our research, gos beyond the compute domain, and identifies the gaps in current storage system designs. We proposes a framework that aligns more closely with the growing data demands. We present a detailed analysis of data movement challenges within the archival workflows and demonstrate how the strategic integration of CSDs can significantly optimize data compression, encryption, as well as other data management tasks, to improve overall system performance. By leveraging the parallel processing capabilities of FPGAs and the high internal bandwidth of SSDs, Salient Store reduces the communication latency and data volume by ~6.2x and ~6.1x, respectively. This paper provides a comprehensive overview of the potential of CSDs to revolutionize storage, making them not just data repositories but active participants in the computational process.
BibTeX
@inproceedings{mishra2024salient,
  author = {Cyan Subhra Mishra and Deeksha Chaudhary and Mahmut Taylan Kandemir and Chita R. Das},
  title = {{Salient Store: Enabling Smart Storage for Continuous Learning Edge Servers}},
  booktitle = {PACT},
  year = {2024},
}
Figure