Signal Processing for Time Domain Wavelengths of Ultra-Weak FBGs Array in Perimeter Security Monitoring Based on Spark Streaming

Zhenhao Yu, Fang Liu, Yinquan Yuan, Sihan Li, Zhengying LiSensors 2018 · DOI

TL;DR

This paper introduces an adaptive parameter DBSCAN (AP-DBSCAN) algorithm integrated with Spark Streaming for real-time perimeter intrusion detection using ultra-weak fiber Bragg gratings (UWFBGs). By automatically determining clustering parameters and leveraging distributed stream processing, the system achieves accurate and efficient anomaly detection for large-scale sensor data.

What Problem Does This Paper Address?

Perimeter security systems are crucial for detecting intrusions, but traditional methods often struggle with the massive volume and high frequency of data generated by modern sensor networks. Specifically, fiber optic sensing technologies like ultra-weak fiber Bragg gratings (UWFBGs) produce rich data streams that require efficient processing for real-time analysis. Existing density-based clustering algorithms, such as DBSCAN, are sensitive to parameter tuning, making them difficult to apply in an unsupervised, real-time setting. This paper addresses the challenge of processing large-scale, high-frequency sensor data for perimeter intrusion detection by developing an adaptive, stream-based clustering approach that automates parameter selection and handles data efficiently.

Methods

The core of this work is the Adaptive-Parameters DBSCAN (AP-DBSCAN) algorithm. Unlike traditional DBSCAN (T-DBSCAN), which requires manual setting of neighborhood parameters (epsilon 'ε' and minimum points 'Nmin'), AP-DBSCAN automatically determines these parameters. It achieves this by:

  • Feature Extraction: Calculating characteristic parameters like "energy" and "average amplitude" from the time-domain wavelength signals of UWFBGs.
  • Normalization: Applying min-max normalization to map these features to a [0,1] range, making them comparable.
  • Distance Matrix Construction: Creating a symmetric distance matrix between all data points.
  • Adaptive Parameter Determination:
    • ε (Epsilon): Determined by analyzing the sorted distances within the characteristic column matrix. The maximum distance in the characteristic column matrix that yields the minimum J value (a measure of cluster quality) is selected as ε.
  • Nmin (Minimum Points): Calculated as the arithmetic mean of points within the ε-neighborhood across the entire dataset.
  • Spark Streaming Integration: The AP-DBSCAN algorithm is implemented on a Spark Streaming platform. This allows for the processing of continuous data streams by dividing them into small batches (e.g., 4 seconds) and processing each batch as a Discretized Stream (DStream) of Resilient Distributed Datasets (RDDs). This enables real-time analysis and handling of large data volumes.
  • Noise Handling: The system incorporates Box-plots and Fast Fourier Transform for noise reduction and data reconstruction, ensuring that normal data is correctly identified and integrated.

Datasets

The paper utilizes data collected from two types of UWFBG sensing arrays:

  • Suspended UWFBG sensing array: A 100m array (20 sensors) fixed along a railing.
  • Buried UWFBG sensing array: A 300m array (60 sensors) buried underground.

For evaluation, simulated intrusion behaviors were generated and recorded:

  • Railing Sensors: Knocking, shaking, and climbing on the railing. Static data was also included.
  • Buried Sensors: Walking on the cable, walking parallel to the cable at distances of 20, 40, and 60 cm, and static standing.

The paper also compares AP-DBSCAN with K-means and Fuzzy C-means (FCM) using various data set sizes (150 kB to 1625 kB) for railing sensors and (131 kB to 1721 kB) for buried sensors.

Key Insights and Ideas

  • Unsupervised Parameter Tuning: The most significant insight is the successful development of an AP-DBSCAN method that automates the selection of crucial clustering parameters (ε and Nmin). This removes the need for manual tuning, which is a major bottleneck for real-time, unsupervised anomaly detection systems.
  • Scalability with Spark Streaming: Integrating AP-DBSCAN with Spark Streaming demonstrates its capability to handle large-scale, high-velocity data streams efficiently. This is crucial for practical deployment in perimeter security where continuous monitoring is required.
  • Effective Intrusion Classification: The AP-DBSCAN method, when applied to the UWFBG data, effectively distinguishes between different types of intrusion behaviors (e.g., knocking vs. climbing, or different walking patterns near buried cables). This is achieved by clustering based on extracted energy and average amplitude features.
  • Performance Equivalence to T-DBSCAN: The AP-DBSCAN method achieves comparable accuracy to the manually tuned T-DBSCAN, indicating that the adaptive parameter selection does not compromise detection performance.

Key Results

  • Accuracy: AP-DBSCAN achieved the same accuracy as T-DBSCAN for both railing and buried sensor data, demonstrating its effectiveness without manual parameter tuning.
  • Error Rate (ER): In comparisons with K-means and FCM, AP-DBSCAN consistently produced lower error rates across various data set sizes for both railing and buried sensors. For example, for a 1625 kB data set of railing sensors, AP-DBSCAN had an ER of 3.0%, compared to 8.1% for K-means and 3.0% for FCM (though FCM was also 3.0% in this specific case, AP-DBSCAN's consistency across different data sizes is notable). For buried sensors, AP-DBSCAN achieved an ER of 1.5% for a 131 kB data set, significantly lower than K-means (7.6%) and FCM (3.8%).
  • Computation Time: While not explicitly detailed as a primary result, the integration with Spark Streaming suggests improved processing speed for large datasets compared to single-machine implementations. Figure 9 shows that for larger datasets (e.g., 12,800 KB), Spark Streaming AP-DBSCAN is significantly faster than single-machine AP-DBSCAN.
  • Intrusion Type Identification: The clustering results (Figures 5 and 7) visually demonstrate the separation of different intrusion behaviors into distinct clusters, confirming the system's ability to classify events.

Limitations and Future Work

  • Limitations:
    • The paper mentions that "there is a small difference between AP-DBSCAN on a single machine and AP-DBSCAN on Spark Streaming when the test data is small." This suggests that for very small datasets, the overhead of Spark Streaming might not offer a significant advantage.
  • While AP-DBSCAN automates parameter selection, the underlying effectiveness still relies on the quality of the extracted features (energy and average amplitude) and the nature of the UWFBG signals.
  • Future Work:
    • The paper's conclusion highlights the system's ability to "detect abnormal conditions of large-scale data in real-time, providing better convenience and service for perimeter security." Future work could focus on further optimizing the algorithm for even higher data rates, exploring more sophisticated feature extraction techniques, and potentially integrating machine learning models for more nuanced intrusion classification. Expanding the types of simulated intrusions and testing in real-world, diverse environmental conditions would also be valuable.

Summary updated

BibTeX
@article{yu2018signal,
  author = {Zhenhao Yu and Fang Liu and Yinquan Yuan and Sihan Li and Zhengying Li},
  title = {{Signal Processing for Time Domain Wavelengths of Ultra-Weak FBGs Array in Perimeter Security Monitoring Based on Spark Streaming}},
  journal = {Sensors},
  year = {2018},
  doi = {10.3390/s18092937},
}
Figure