Enhanced Data-Driven Product Development via Gradient Based Optimization and Conformalized Monte Carlo Dropout Uncertainty Estimation

Andrea Thomas Nava, Lijo Johny, Fabio Azzalini, Johannes Schneider, Arianna Casanova

TL;DR

This paper introduces a robust framework for Data-Driven Product Development (DDPD) that uses Projected Gradient Descent on multi-output neural networks to optimize multiple correlated product properties simultaneously while avoiding extreme extrapolation. To ensure these optimized designs are reliable, the authors develop Conformalized Monte Carlo Dropout (ConfMC), a novel uncertainty quantification method that provides mathematically guaranteed, adaptive prediction intervals without requiring model retraining when confidence levels change.

What Problem Does This Paper Address?

In manufacturing and engineering, designing a new product traditionally relies on expensive, slow, and iterative physical prototyping. Data-Driven Product Development (DDPD) aims to accelerate this by training machine learning models on historical experimental data to predict product properties from design specifications.

However, standard DDPD approaches face three major hurdles:

  1. Multi-objective, correlated targets: Real-world products rarely have just one metric to optimize. For instance, a structural component might need to maximize both strength and flexibility, two properties that are inherently correlated.
  2. Extrapolation risks: Optimization algorithms can easily propose unrealistic, unmanufacturable designs that lie far outside the training data distribution.
  3. Unreliable Uncertainty Quantification (UQ): Standard deep learning models are notoriously overconfident. Traditional UQ methods like Bayesian Neural Networks or standard Monte Carlo (MC) Dropout either lack finite-sample guarantees or fail to adapt to data heterogeneity without heavy computational costs.

Methods

The authors tackle these challenges by combining gradient-based optimization with a clever twist on conformal prediction.

1. Constrained Multi-Target Optimization

Instead of using slow derivative-free methods like Bayesian Optimization, the framework trains a multi-output Multi-Layer Perceptron (MLP) to jointly model all target properties. Because neural networks are differentiable, the authors use Projected Gradient Descent (PGD) to compute gradients with respect to the input features themselves, effectively backpropagating through the network to alter the product design toward an optimal target. To keep the designs realistic, they introduce:

  • Hyper-rectangle constraints: Restricting the search space to user-defined feasible ranges.
  • Gradient masking: Allowing users to freeze specific features (like fixed environmental testing conditions) while optimizing others.

2. Conformalized Monte Carlo Dropout (ConfMC)

To provide trustworthy uncertainty estimates, the authors introduce ConfMC. Standard MC Dropout generates a heuristic predictive distribution by keeping dropout active during inference, but it lacks statistical guarantees and frequently undercovers true values.

ConfMC fixes this by applying Nested Conformal Prediction to the quantiles generated by MC Dropout. Instead of relying on a fixed nominal error rate ($\alpha$), ConfMC evaluates a sequence of prediction intervals on a held-out calibration dataset to find the exact threshold that guarantees the desired coverage. This yields two massive benefits:

  • Adaptivity: The prediction intervals vary in width depending on how certain the model is about a specific region of the design space.
  • No Retraining: Unlike Conformal Quantile Regression (CQR), which requires retraining the underlying model if a user wants to change the confidence level, ConfMC allows users to dynamically adjust coverage levels on the fly.

Datasets

The authors evaluated the uncertainty quantification performance of ConfMC against four baseline methods across five publicly available datasets (summarized in Table 1):

  • Concrete: 1,030 rows, 8 features (predicting compressive strength).
  • Kernel: 241,600 rows, 14 features.
  • Popularity: 39,644 rows, 59 features.
  • Superconduct.: 21,263 rows, 81 features.
  • BlogData: 52,397 rows, 280 features.
Table 1: Datasets used for experimental validation.

Key Insights and Ideas

  • Inverting the Network for Design: Treating the input space as the optimization target via backpropagation allows for highly efficient navigation of complex design spaces compared to traditional population-based or evolutionary algorithms.
  • The Power of Calibration: Heuristic uncertainty methods (like MC Dropout) can be systematically corrected using conformal prediction. By nesting these intervals, you get the "best of both worlds": the data-driven flexibility of deep learning and the rigorous mathematical guarantees of conformal inference.
  • Dynamic User Control: In industrial settings, engineers need to toggle confidence levels based on risk tolerance. ConfMC’s ability to adjust these intervals instantly without retraining the underlying neural network is a major win for practical usability.

Key Results

Uncertainty Quantification Performance

The framework was tested with a target empirical coverage of 80% ($\alpha = 0.2$).

  • Coverage Reliability: As shown in Table 2, standard MC Dropout severely undercovered the true values (dropping as low as 45-53%). NGBoost also showed large deviations. In contrast, ConfMC consistently hit or neared the 80% target across all datasets.
  • Interval Efficiency: According to Table 3, ConfMC provides a balanced trade-off, generating adaptive, variable-width intervals that capture data heterogeneity without becoming excessively wide.
Table 2: Average Empirical Coverage (AEC) for α = 0.2 across the 20 runs.
Table 3: Average Interval Width (AIW) for α = 0.2 across the 20 runs.

This adaptivity is visually striking in Figure 1, where standard Conformal Prediction (CP) produces rigid, constant-width intervals, while ConfMC narrows or widens its bounds based on localized data density.

Figure 1: Empirical Y-conditional coverage. MC prediction intervals severely undercover, standard CP produces constant- width PIs, while CQR and Conf-MC generate adaptive, variable-width intervals.
Figure 1: Empirical Y-conditional coverage. MC prediction intervals severely undercover, standard CP produces constant- width PIs, while CQR and Conf-MC generate adaptive, variable-width intervals.

Industrial Deployment

The framework was deployed via an interactive Streamlit web application (shown in Figure 2) at a leading construction tools manufacturing company to optimize diamond segment prototypes. User surveys indicated that the tool directly led to fewer physical prototypes, faster development cycles, and reduced resource waste, while the reliable UQ intervals helped build crucial trust with design engineers.

Figure 2: Web interface implementing the proposed method with user-selected coverage and error tolerance. Blue dots: tested products; green dot: search start; red dot: search end; red rectangle: prediction intervals at the chosen error rate.

Limitations and Future Work

The primary limitation of this framework is its reliance on the assumption of data exchangeability (a core requirement for conformal prediction). In practice, as an optimization algorithm steers designs toward high-performance regions, it can introduce feedback covariate shift, meaning the newly proposed prototypes come from a different distribution than the historical training data.

While the authors attempt to mitigate this using PGD constraints and wider MC dropout intervals for out-of-distribution zones, they note that a rigorous investigation into handling feedback covariate shift in DDPD remains an important avenue for future research. Additionally, incorporating automatically generated explanations for why a certain prototype is recommended could further enhance real-world adoption.

Summary updated

Abstract

Data-Driven Product Development (DDPD) leverages data to learn the relationship between product design specifications and resulting properties. To discover improved designs, we train a neural network on past experiments and apply Projected Gradient Descent to identify optimal input features that maximize performance. Since many products require simultaneous optimization of multiple correlated properties, our framework employs joint neural networks to capture interdependencies among targets. Furthermore, we integrate uncertainty estimation via \emph{Conformalised Monte Carlo Dropout} (ConfMC), a novel method combining Nested Conformal Prediction with Monte Carlo dropout to provide model-agnostic, finite-sample coverage guarantees under data exchangeability. Extensive experiments on five real-world datasets show that our method matches state-of-the-art performance while offering adaptive, non-uniform prediction intervals and eliminating the need for retraining when adjusting coverage levels.
BibTeX
@misc{nava2026enhanced,
  author = {Andrea Thomas Nava and Lijo Johny and Fabio Azzalini and Johannes Schneider and Arianna Casanova},
  title = {{Enhanced Data-Driven Product Development via Gradient Based Optimization and Conformalized Monte Carlo Dropout Uncertainty Estimation}},
  year = {2026},
  eprint = {2601.00932},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.00932},
}
Figure