Abstract
In the manufacturing industry, identifying the range of input conditions under which a machine operates properly often requires expensive testing. Because it is difficult to accurately control input conditions during actual usage, machine performance must be guaranteed by incorporating potential variations. We formulate this scenario as an Input Uncertain Reliable Level Set Estimation (IU-rLSE) problem. The goal is to identify input ranges where outputs meet a desired threshold with high probability under uncertainty. We propose an efficient active learning method, provide theoretical convergence analysis, and demonstrate its performance on synthetic and real-world data.
Introduction
In high-stakes engineering, the "Level Set" represents the range of inputs (e.g., temperature, pressure) where a system performs correctly. Traditional Level Set Estimation (LSE) assumes precise control over these inputs. However, in real-world manufacturing, inputs often fluctuate. A setting intended to be 100°C might vary between 98°C and 102°C. If an algorithm ignores this noise, it may select operating points that are safe on average but fail frequently due to these small variations.
The IU-rLSE Framework
To address this, we introduce IU-rLSE. This framework identifies regions where the probability of success remains high even when accounting for input uncertainty.
As shown in the figure above, standard LSE (a) might identify points as "lower than threshold $h$" without considering the risk posed by nearby unsafe regions. IU-rLSE explicitly models the input uncertainty (the distribution curves at the bottom of panels b and c). A point is only classified as reliable (c) if the output remains below $h$ with high probability across the entire noise distribution.
Methodology
The proposed algorithm uses Gaussian Process (GP) modeling to create a probabilistic map of the unknown function. The core innovation lies in the acquisition function:
- Reliability Metric: Instead of mean performance, we optimize for the probability that the output stays within safe bounds.
- Computational Efficiency: Calculating expected improvement under uncertainty typically involves complex triple integrals. We developed a mathematical approximation that reduces this to a single integral, making the method practical for real-time experimental design.
- Adaptive Noise Estimation: The method can estimate unknown input noise parameters on the fly during the testing process.
Experimental Evaluation
We evaluated IU-rLSE against standard benchmarks and the Combined Cycle Power Plant (CCPP) dataset.
- Superior Accuracy: IU-rLSE achieved higher F1-scores than traditional LSE methods like Straddle or MILE across various noise profiles (Gaussian and Gamma).
- Sample Efficiency: On real-world power plant data, the method identified reliable operating regions significantly faster, requiring fewer expensive data points.
- Theoretical Guarantees: We proved that the algorithm is mathematically guaranteed to converge to the correct reliable set over time.
Conclusion
IU-rLSE provides a robust framework for identifying safe operating regions in the presence of uncontrollable input noise. By combining high-probability safety guarantees with a computationally efficient active learning strategy, it offers a practical tool for modern manufacturing and engineering optimization.
Summary updated