Abstract
Predictive Monitoring (PM) is a vital safety mechanism for cyber-physical systems, yet traditional methods often assume full state observability. This paper introduces a robust framework for Neural Predictive Monitoring under Partial Observability (NPM-PO), utilizing deep learning to forecast safety violations from noisy and incomplete sensor data. We propose two distinct architectures—an end-to-end model and a modular two-step estimator-classifier—and augment them with Conformal Prediction (CP) to provide statistically grounded reliability metrics. Our evaluation across six non-linear benchmarks demonstrates that an uncertainty-based rejection criterion can achieve near-perfect error detection, while an active learning loop significantly improves prediction efficiency and accuracy.
Introduction
In safety-critical domains such as autonomous driving and medical device control, systems must anticipate and avoid future constraint violations. This is the core objective of Predictive Monitoring (PM). However, real-world systems rarely provide a perfect view of their internal state; sensors are inherently noisy, and many critical variables remain hidden. This condition, known as Partial Observability (PO), often leads standard neural network monitors to produce overconfident but incorrect safety predictions.
This paper addresses the challenge of making PM both accurate and reliable under "foggy" conditions. By combining deep learning with the statistical framework of Conformal Prediction, we provide not only safety forecasts but also mathematically grounded "reliability scores" that can preemptively flag potential prediction errors.
Methodological Approaches
We extend the Neural Predictive Monitoring (NPM) framework to handle partial observability through two primary architectural strategies:
- End-to-End Approach: A Convolutional Neural Network (CNN) is trained to map raw, noisy observation sequences directly to a safety prediction (i.e., whether the system will enter an unsafe set $U$ within a future horizon $H_f$).
- Two-Step Approach: This modular strategy first employs a Neural State Estimator (NSE) to reconstruct the hidden system state from noisy observations. A subsequent Neural State Classifier (NSC) then predicts safety based on this reconstructed state. A critical fine-tuning phase follows, where both networks are optimized jointly to ensure the classifier learns to compensate for the specific estimation errors produced by the NSE.
The Predictive Monitoring Pipeline
To ensure these predictions are trustworthy, the framework integrates Conformal Prediction (CP). Unlike standard classifiers that output a single label, CP produces prediction regions guaranteed to contain the true value with a user-defined probability (e.g., 95%). From these regions, we derive "Confidence" and "Credibility" metrics.
These metrics feed into an uncertainty-based rejection criterion, typically implemented as a Support Vector Classifier (SVC). This allows the system to identify and "reject" predictions that are likely to be erroneous, effectively allowing the monitor to say "I don't know" when data is too ambiguous. Furthermore, an Active Learning loop uses these rejected points to selectively retrain the model, tightening prediction regions and improving efficiency without sacrificing safety guarantees.
Experimental Evaluation
The framework was evaluated on six diverse hybrid system benchmarks, ranging from a 2D Inverted Pendulum (IP) to a 28D Helicopter Controller (HC). The benchmarks included highly non-linear dynamics such as Spiking Neurons (SN), Coupled Van Der Pol oscillators (CVDP), and Triple Water Tanks (TWT).
Key Results
- High Accuracy: Reachability predictions exceeded 99% accuracy across most case studies, even in the presence of significant sensor noise.
- Reliable Error Detection: The rejection mechanism proved highly effective, often reaching a 100% error detection rate, ensuring that almost no incorrect safety predictions were accepted by the system.
- Real-Time Performance: Despite the underlying complexity, a single prediction takes between 0.7 and 0.9 microseconds on a GPU, making the framework suitable for high-frequency real-time deployment.
- State Estimation Superiority: The Neural State Estimator significantly outperformed traditional filtering methods, such as Unscented Kalman Filters (UKF) and Moving Horizon Estimation (MHE), particularly on non-linear benchmarks.
Discussion and Insights
A primary finding of this research is that the Two-Step approach generally outperforms the End-to-End mapping. By forcing the model to explicitly reconstruct the system state, we provide a better inductive bias rooted in the underlying physics of the system.
Additionally, the study highlights the utility of uncertainty as a proactive tool. The integration of Conformal Prediction allows for a rigorous definition of "reliability," which in turn enables effective Active Learning. While the theoretical guarantees of CP assume data exchangeability—a condition often violated by sequential time-series data—empirical results showed that the 95% confidence intervals remained strictly honored in practice.
Conclusion and Future Work
This work demonstrates that reliable predictive monitoring is achievable even under partial observability. Future research will explore the use of deep generative models, such as Variational Autoencoders (VAEs), to handle fully stochastic system dynamics. Additionally, investigating architectures like Transformers or Recurrent Neural Networks (RNNs) may further improve the capture of long-term temporal dependencies in sequential sensor data.
Summary updated