Abstract
In the manufacturing industry, it is often necessary to repeat expensive operational testing of machine in order to identify the range of input conditions under which the machine operates properly. Since it is often difficult to accurately control the input conditions during the actual usage of the machine, there is a need to guarantee the performance of the machine after properly incorporating the possible variation in input conditions. In this paper, we formulate this practical manufacturing scenario as an Input Uncertain Reliable Level Set Estimation (IU-rLSE) problem, and provide an efficient algorithm for solving it. The goal of IU-rLSE is to identify the input range in which the outputs smaller/greater than a desired threshold can be obtained with high probability when the input uncertainty is properly taken into consideration. We propose an active learning method to solve the IU-rLSE problem efficiently, theoretically analyze its accuracy and convergence, and illustrate its empirical performance through numerical experiments on artificial and real data.
Introduction: Beyond Standard Level Set Estimation
In many engineering and manufacturing contexts, identifying the "safe" operating region of a system is critical. Traditionally, this is framed as Level Set Estimation (LSE), where the goal is to find the set of inputs $x$ such that the system output $f(x)$ remains below a threshold $h$. However, standard LSE assumes perfect control over inputs. In practice, inputs are often subject to stochastic fluctuations—input uncertainty.
A point that appears safe in a deterministic setting might become unreliable if small deviations in the input push the system into a failure state. This paper introduces Input Uncertain Reliable Level Set Estimation (IU-rLSE) to address this. Instead of finding where $f(x) \leq h$, IU-rLSE seeks regions where the probability of success remains high (e.g., $> \alpha$) despite input noise.
Methodology: The IU-rLSE Framework
The authors model the unknown system function $f$ using Gaussian Processes (GP). The core of the IU-rLSE approach lies in estimating the reliability probability $p_x^* = P(f(s) \leq h | s \sim g(\cdot|x))$, where $g$ is the distribution of input uncertainty around a target $x$.
Key Innovations:
- Reliability-Based Acquisition: The method employs an active learning strategy to select the next experimental point. It adapts the "Maximum Improvement for Level-set Estimation" (MILE) function to target the boundary of the reliable region rather than the function's threshold boundary.
- Computational Efficiency: Calculating the acquisition function typically involves triple-nested integrals, leading to $O(M^3)$ complexity. The authors introduce a Taylor-like approximation around the expected input, reducing the cost to $O(M)$ and making the algorithm practical for real-time use.
- Theoretical Guarantees: The paper provides convergence proofs, ensuring that the algorithm will correctly identify the reliable set with high probability as the number of iterations increases.
Experimental Results
The IU-rLSE algorithm was benchmarked against standard active learning methods like Straddle and MILE on synthetic functions and the real-world Combined Cycle Power Plant (CCPP) dataset.
The results demonstrate that standard LSE methods fail to find truly reliable regions because they optimize for the wrong target. In contrast, IU-rLSE consistently achieves significantly higher F1-scores and precision. On the CCPP dataset, IU-rLSE reached a precision of 1.0, while competing methods struggled to maintain reliability under noise.
Conclusion and Future Directions
IU-rLSE provides a robust framework for safety-critical applications where input precision cannot be guaranteed. By shifting the focus from function values to reliability probabilities and introducing a computationally efficient acquisition function, the authors bridge the gap between theoretical LSE and practical manufacturing needs.
Future work may address scenarios with completely unknown or non-stationary noise distributions, as well as scaling the GP models to higher-dimensional input spaces using sparse approximations.
Summary updated