STeP-C: Real-Time Neural Compression for Distributed Fiber-Optic Sensing at Scale

Tingfeng Li, Ming-Fang Huang, Shaobo Han, Yue TianarXiv 2026 · arXiv

TL;DR

STeP-C is the first neural compression framework capable of handling the massive, real-time data streams generated by kilometer-scale fiber-optic sensing. By combining a spatio-temporal predictive architecture with a highly optimized, GPU-accelerated entropy coding pipeline, it achieves a 99% reduction in data volume while maintaining the signal integrity required for automated event detection.

What Problem Does This Paper Address?

Distributed Acoustic Sensing (DAS) is a revolutionary technology that turns standard telecommunications fiber-optic cables into thousands of individual vibration sensors. By sending laser pulses down the fiber and measuring backscatter, researchers can monitor everything from traffic patterns and pipeline intrusions to seismic activity across entire cities or subsea cables.

The catch? The data volume is staggering. A single 81 km fiber sampled at 1 kHz generates hundreds of megabytes per second—over a terabyte every hour. This creates a massive bottleneck for real-time transmission and long-term storage. While standard image and video codecs (like JPEG 2000 or H.264) exist, they aren't optimized for the unique "anisotropic" structure of DAS data, where signals are highly correlated along the 1D path of the fiber but evolve differently over time. Previous neural compression attempts were either too slow for live streaming or required constant, expensive retraining.

Methods

The authors introduce STeP-C (Spatio-Temporal Predictive Compression). The architecture follows a "predict-then-code" philosophy:

  1. Spatial Encoding: A 1D convolutional neural network (CNN) compresses the raw data from thousands of fiber channels into a compact "latent" representation.
  2. Temporal Prediction: An LSTM (Long Short-Term Memory) network looks at past frames to predict what the next latent frame will look like.
  3. Residual Coding: Instead of coding the whole frame, the system only codes the residual—the difference between the prediction and the actual measurement. This is much more efficient because DAS signals are often highly predictable over short time windows.
Figure 1. STeP-C for real-time DAS compression. (a) A telecom fiber acts as a dense array of sensing channels. (b) The DAS readout forms a spatio-temporal “video” in which physical events appear as structured patterns. (c) STeP-C predicts each frame and entropy-codes only the residual. (d) Reconstructed signals yield consistent downstream detections.

The real breakthrough, however, is in the Systems Engineering. The authors identified that the primary bottleneck wasn't the neural network itself, but the "entropy coding" (the final step of turning numbers into a bitstream). To fix this, they moved symbol computation to the GPU, implemented parallel, GIL-free arithmetic coding in C++, and used FP16 (half-precision) inference.

Datasets

The model was evaluated on three real-world datasets:

  • NJCFA (NJ Coastal Fiber Array): 3,584 channels along the New Jersey coast, used for benchmarking against traditional codecs.
  • DADAS (Dallas Ambient DAS): An urban deployment in Dallas with 81,000 channels, used to test how well the model generalizes to unseen parts of a fiber cable.
  • PFT (Princeton Fiber Testbed): A live, 81,000-channel stream used to validate the end-to-end real-time pipeline.

Key Insights and Ideas

  • The Entropy Bottleneck: In many neural compression papers, the time taken for entropy coding is ignored because evaluations are done offline. STeP-C reveals that for real-world streaming, classical arithmetic coding is the "silent killer" of throughput, requiring significant low-level optimization to keep up with hardware.
  • Generalization without Retraining: A major concern with neural codecs is that they might only work on the specific data they were trained on. STeP-C demonstrates that a model trained on one 4,000-channel segment of a cable can effectively compress other segments of the same 81,000-channel cable without needing to be retrained.
  • Preserving "Machine Vision" Quality: Unlike video codecs designed to look good to human eyes, DAS compression must preserve the mathematical features used by downstream AI detectors. STeP-C was specifically tested to ensure that an automated event detector would reach the same conclusions on compressed data as it would on the original.

Key Results

  • Massive Compression: Achieved an 87.3x compression ratio (a 98.85% reduction in file size).
  • High Fidelity: Maintained a Peak Signal-to-Noise Ratio (PSNR) of 31.04 dB, significantly outperforming H.264 and MP3 at similar bitrates.
  • Blazing Speed: The optimized pipeline reached a throughput of 535 MB/s, comfortably exceeding the 324 MB/s requirement for real-time 81 km DAS monitoring.
  • Detector Consistency: Even at high compression levels, the system maintained a 96.62% agreement with high-confidence event detections from the original uncompressed data.
Throughput Gains from Optimizations

Limitations and Future Work

While STeP-C generalizes well across different sections of the same cable, the authors note that future work is needed to see how well a model trained on one geographic location (e.g., a city) performs on a completely different environment (e.g., a subsea cable). Additionally, they aim to extend validation to more complex tasks like precise event localization and "arrival-time picking" for seismic analysis.

Summary updated

Abstract

Distributed fiber-optic sensing (DFOS) technologies such as Distributed Acoustic Sensing (DAS) generate continuous, high-resolution spatio-temporal data at rates exceeding hundreds of megabytes per second, creating severe challenges for real-time storage and transmission. While neural compression methods can achieve strong rate-distortion performance on sensing data, prior approaches either require per-segment retraining incompatible with streaming, or have only been validated offline. None have been demonstrated end-to-end on a continuously streaming DAS acquisition system at scale. We present STeP-C (Spatio-Temporal Predictive Compression), a neural compression framework that decouples spatial encoding, temporal prediction, and entropy-optimized residual coding, trained end-to-end with a rate-distortion objective motivated by conditional entropy reduction. Critically, we identify entropy coding as the dominant bottleneck for streaming deployment. We address it through GPU-side symbol computation, cached probability tables, and parallel GIL-free arithmetic coding, together with FP16 inference and batched entropy coding, enabling a full real-time pipeline built around a GPU ring buffer. We validate this pipeline on a live DAS testbed, achieving a 98.85% file size reduction at 31.04 dB full-band PSNR and an average throughput of 535 MB/s, exceeding the 324 MB/s requirement for 1 kHz, 81 km DAS streams. We further show that STeP-C substantially outperforms conventional codecs at practical streaming bitrates, can be applied to unseen channel ranges within the same acquisition system without retraining, and largely maintains downstream detector consistency, with 96.62% high-confidence class agreement and 100% Jaccard similarity on detected event types.
BibTeX
@misc{li2026step,
  author = {Tingfeng Li and Ming-Fang Huang and Shaobo Han and Yue Tian},
  title = {{STeP-C: Real-Time Neural Compression for Distributed Fiber-Optic Sensing at Scale}},
  year = {2026},
  eprint = {2611.17126},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17126},
}
Figure