Abstract: Various real-world applications involve modeling complex systems with immense uncertainty and optimizing multiple objectives based on the uncertain model. Quantifying the impact of the model uncertainty on the given operational objectives is critical for designing optimal experiments that can most effectively reduce the uncertainty that affect the objectives pertinent to the application at hand. In this paper, we propose the concept of mean multi-objective cost of uncertainty (multi-objective MOCU) that can be used for objective-based quantification of uncertainty for complex uncertain systems considering multiple operational objectives. We provide several illustrative examples that demonstrate the concept and strengths of the proposed multi-objective MOCU. Furthermore, we present a real-world example based on the mammalian cell cycle network to demonstrate how the multi-objective MOCU can be used for quantifying the operational impact of model uncertainty when there are multiple, possibly competing, objectives.
Introduction
In complex engineering and biological systems, models are rarely perfect. We typically operate within an "uncertainty class" of potential models. Traditional metrics like entropy or variance quantify uncertainty in an "operation-agnostic" manner—they measure how much we don't know without considering if that lack of knowledge actually affects our decisions.
The Mean Objective Cost of Uncertainty (MOCU) was previously introduced to bridge this gap for single-objective problems. It measures the performance degradation caused by uncertainty: the difference between a robust strategy (the best "safe bet") and the ideal strategy (what one would do with perfect information). However, real-world applications—such as balancing drug efficacy against side effects or material strength against weight—require optimizing multiple, often competing, objectives. This paper extends the MOCU framework to these multi-objective scenarios.
Methodology
The proposed multi-objective MOCU framework transforms a complex multi-objective problem into a single quantifiable cost of uncertainty through a three-step process:
- Weighted Cost Aggregation: Multiple objectives are combined into a single cost function using a weight vector ($\lambda$).
- Calculating the Performance Gap: For any specific weight distribution, the framework calculates the gap between the performance of an optimal robust operator and the model-specific optimal operator.
- Averaging over Priorities: Since the relative importance of objectives is often unknown or subjective, the "mean multi-objective MOCU" integrates this cost across all possible weight distributions (typically using a Dirichlet distribution).
This approach ensures that the metric represents the total operational impact of uncertainty, regardless of how a future decision-maker might prioritize specific goals.
Experimental Validation
The framework was validated using two distinct setups:
- Synthetic Quadratic Test Functions: These generalized benchmarks allow for precise control over how global minima shift under uncertainty, providing a clear view of MOCU's behavior compared to traditional metrics.
- Mammalian Cell Cycle Network: A 10-gene Boolean Network with perturbation (BNp) representing a real-world biological system. This model simulates therapeutic interventions where the goal is to minimize cancerous states while avoiding pathological collateral damage.
Key Results and Insights
A primary insight is that uncertainty is highly context-dependent. A model might have high entropy, but if that uncertainty doesn't change the optimal decision, the MOCU remains zero. MOCU effectively filters out irrelevant noise, focusing only on uncertainty that "matters."
In synthetic tests, MOCU proved highly sensitive to parameters that significantly impact control, whereas entropy and variance remained flat. For example, as shown in the 3D analysis of the synthetic model, MOCU scales differently depending on which parameters ($c$ or $d$) are uncertain.
In the biological cell cycle simulation, the multi-objective MOCU increased as more regulatory "edges" in the network became unknown. Interestingly, the results indicated that a few "critical" edges caused a disproportionately high cost of uncertainty, while many others had negligible impact. This makes multi-objective MOCU an ideal "compass" for Optimal Experimental Design (OED), identifying which experiments will most effectively improve decision-making capabilities.
Conclusion
The multi-objective MOCU framework provides a robust, objective-based metric for quantifying the impact of model uncertainty in complex systems. While computationally intensive due to nested optimizations and integrations, it offers a superior alternative to traditional uncertainty measures for practical engineering tasks. Future work will focus on integrating this metric into Bayesian Optimization to more efficiently estimate Pareto fronts and applying it to active learning in fields like drug discovery and materials science.
Summary updated