Optimal experimental design: from design point to design region

Martin Bubel, Philipp Seufert, Gleb O. Karpov, Jan Schwientek, Michael Bortz, Ivan Oseledets

TL;DR

This paper moves beyond traditional "point-based" optimal experimental design (OED) by introducing methods to calculate optimal design regions. By accounting for uncertainty in model parameters, the authors provide researchers with "confidence ellipsoids" in the design space, offering a more robust and flexible roadmap for conducting experiments in nonlinear systems.

What Problem Does This Paper Address?

In fields like chemical engineering, researchers use Optimal Experimental Design (OED) to determine the best conditions (e.g., temperature, pressure) to run experiments to calibrate mathematical models. The goal is to maximize the information gained while minimizing costs.

However, there is a "chicken and egg" problem: the optimal design for a nonlinear model depends on the very parameters ($\theta$) you are trying to estimate. If your initial guess for $\theta$ is slightly off, your "optimal" design points might be completely wrong. Traditional approaches try to find a single robust design (like "average-case" or "worst-case" optimization), but these often hide the underlying sensitivity of the design or require massive computational power. This paper addresses the need for a method that shows researchers not just where to measure, but how much that "where" shifts due to uncertainty.

Methods

The authors propose two distinct strategies to transition from discrete design points to design regions:

  1. Global Clustering Approach:
    • The Workflow: They use Monte Carlo sampling to generate many possible parameter scenarios. For each scenario, they calculate the optimal design.
  2. The Innovation: Instead of just averaging these, they use the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm. DBSCAN identifies regions in the design space where optimal points frequently "cluster."
    • Why it works: Unlike the common k-means algorithm, DBSCAN doesn't require you to guess the number of clusters beforehand and can account for the "weight" (importance) of each design point.
  3. Local Approximation Approach:
    • The Workflow: This method stays close to a "nominal" (best guess) parameter set. It uses the Implicit Function Theorem and first-order derivatives to perform a sensitivity analysis.
  4. The Innovation: By calculating how the optimal design points and their weights change relative to $\theta$, they can draw "confidence ellipsoids" around the nominal design points.
  5. Why it works: It is computationally much cheaper than the global approach because it only requires solving one optimization problem and calculating local derivatives.

Datasets

The methods were evaluated using three benchmark cases:

  • Case 1: Exponential Function ($f_1$): A simple 1D design space used to demonstrate how design points "smear" into intervals under uncertainty.
  • Case 2: Exponential-Power Model ($f_2$): A more complex model where the number of optimal design points actually changes (appears/disappears) as parameters shift.
  • Case 3: The "Flash" Model: A real-world chemical engineering application involving vapor-liquid equilibrium for a methanol-water mixture. This uses the highly nonlinear NRTL (Non-Random Two-Liquid) model.

Key Insights and Ideas

  • Design "Smearing": The paper visualizes how uncertainty causes a single optimal point to spread out into a "cloud" or region. Understanding this spread helps an experimenter know if a specific setting is "stable" or highly sensitive to assumptions.
  • Weight Stability: It’s not just the location of the experiment that matters, but its importance (weight). The local approach can predict when an experimental condition might become useless (weight drops to zero) if the true parameters differ from the guess.
  • The Nonlinearity Trap: The "Flash" example revealed that for highly nonlinear models, the local approximation can fail spectacularly, producing "confidence regions" that are physically impossible. This highlights exactly when researchers must switch to the more expensive global clustering method.

Key Results

  • Clustering Success: The DBSCAN approach successfully identified robust regions across all examples, even when the number of design points fluctuated. In the "Flash" model, the clustering approach achieved a 94% coverage rate for half of the samples.
  • Efficiency vs. Accuracy: The local approximation proved excellent for simpler models, providing intuitive ellipsoids with minimal compute. However, it struggled with the NRTL model's high nonlinearity, where the variance was too large for linear approximations to hold.
  • Practical Utility: The authors provide a clear framework for experimenters: start with the local approach to gauge sensitivity; if the regions look "exploded" or the model is known to be highly nonlinear, move to global clustering.

Limitations and Future Work

  • The "Appearance" Problem: While the local method can predict when a design point might disappear (weight goes to zero), it cannot currently predict when a brand-new design point might appear elsewhere in the design space.
  • Dimensionality: The clustering approach, while robust, faces the "curse of dimensionality." Sampling high-dimensional parameter spaces remains a significant computational hurdle.
  • Hyperparameter Tuning: DBSCAN requires choosing a search radius ($r$) and minimum weight ($\omega_{min}$). The authors admit there is no "universal rule" for these, and they must be tuned per problem.

Summary updated

Abstract

Abstract Optimal experimental designs are used in chemical engineering to obtain precise mathematical models. The optimal design consists of design points with a maximal amount of information and thus lead to more precise models than statistical designs. In general, the optimal design depends on an uncertain estimate of unknown model parameters $$\theta $$ . The optimal designs are therefore also uncertain and continuously shift in the design space, as the value of $$\theta $$ changes. We present two approaches to capture this behavior when computing optimal designs, a global clustering approach and a local approximation of the confidence regions. Both methods find an optimal design and assign the optimal design points confidence regions which can be used by an experimenter to decide which design points to use. The clustering approach requires a Monte Carlo sampling of the uncertain parameters and then identifies regions of high weight density in the design space. The local approximation of the confidence regions is obtained via an error propagation using the derivatives of the optimal design points and weights. We apply the introduced approaches to mathematical examples as well as to an application example modeling vapor-liquid equilibria.
BibTeX
@inproceedings{bubel2025optimal,
  author = {Martin Bubel and Philipp Seufert and Gleb O. Karpov and Jan Schwientek and Michael Bortz and Ivan Oseledets},
  title = {{Optimal experimental design: from design point to design region}},
  booktitle = {Statistical},
  year = {2025},
  doi = {10.1007/s00362-025-01725-7},
}

Loading related content...

Figure