Construction of uniform designs over continuous domain in computer experiments

Jianfa Lai, Kai-Tai Fang, Xiaoling Peng, Yuxuan LinCommun. Stat. Simul. Comput 2024 · OpenReview

TL;DR

This paper introduces a novel method for constructing "uniform designs" (UDs) for computer experiments, specifically addressing the challenge of continuous input variables. By combining a threshold accepting algorithm for initialization with coordinate descent optimization, they achieve UDs with improved uniformity and better performance in modeling compared to existing methods.

What Problem Does This Paper Address?

Computer experiments are crucial for understanding complex systems where direct physical experimentation is infeasible. These experiments involve simulating a system with various input parameters to observe the output. A key challenge is selecting the optimal set of input points (the "design") to efficiently explore the input space and build an accurate predictive model (a "metamodel"). Traditional methods often focus on discrete input levels, but real-world factors are frequently continuous. Existing uniform design methods, while effective for discrete domains, struggle with continuous inputs, leading to potential information loss. This paper aims to bridge this gap by developing UDs that are truly uniform over continuous domains, thereby improving the accuracy of subsequent modeling.

Methods

The core of this work lies in a two-stage approach for constructing uniform designs over continuous domains:

  1. Initialization with Threshold Accepting (TA): The authors leverage the Threshold Accepting (TA) algorithm, a metaheuristic optimization technique, to generate an initial "U-type" design. TA is chosen for its ability to escape local optima and its relative efficiency compared to simulated annealing. This initial design serves as a starting point for the subsequent optimization.
  2. Optimization with Coordinate Descent: The initialized design is then refined using coordinate descent algorithms. The objective is to minimize the Centered L2-discrepancy (CD), a measure of uniformity. Three variants of coordinate descent are explored:
  3. Coordinate Gradient Descent (CGD): A standard approach that iteratively adjusts design points along coordinate directions based on the gradient of the CD function.
  4. Coordinate Zero-gradient (CZG): An optimization that sets the coordinate gradient to zero to find optimal values, particularly useful when explicit gradient expressions are complex or when seeking faster convergence.
  5. Coordinate Descent with Fixed Step Size (CDFSS): This method uses a fixed step size, making it suitable for experiments with inherent minimum units or limitations in variable precision.

The paper emphasizes that using a good initialization (from TA) is crucial for the performance of these coordinate descent methods due to the non-convex nature of the CD objective function.

Datasets

The paper does not introduce new datasets. Instead, it evaluates the proposed uniform designs on established benchmark functions commonly used in computer experiments:

  • Wood Model: A well-known function known for its optimization difficulty.
  • Six-hump Camelback Model: Another challenging function with multiple local minima.

The performance of the newly generated uniform designs is compared against existing uniform designs (recorded on a website) and standard sampling methods like Latin Hypercube Sampling (LHS) and Mid-point LHS (MLHS).

Key Insights and Ideas

  • Continuous Domains are Crucial: The paper strongly advocates for the necessity of uniform designs specifically tailored for continuous input spaces, highlighting the limitations of traditional lattice-based designs.
  • TA as a Powerful Initializer: The Threshold Accepting algorithm proves effective in generating a good starting point for uniform designs, which is critical for the success of subsequent optimization steps in non-convex landscapes.
  • Coordinate Descent Variants Offer Flexibility: The exploration of CGD, CZG, and CDFSS demonstrates that different coordinate descent strategies can be employed, each with its own advantages in terms of speed, complexity, and applicability to specific experimental constraints.
  • Improved Modeling Performance: The core insight is that UDs constructed for continuous domains lead to demonstrably better performance in building predictive models (metamodels) compared to existing methods, as evidenced by lower Mean Squared Errors (MSE).

Key Results

The paper demonstrates that the proposed method for constructing uniform designs over continuous domains yields significant improvements:

  • Reduced Discrepancy: The newly generated UDs exhibit lower Centered L2-discrepancy (CD) values compared to existing designs, indicating better uniformity. For instance, Table 3 shows that TA+CGD designs achieve lower CD values than TA-only designs.
  • Enhanced Modeling Accuracy: In case studies using the Wood Model and the Six-hump Camelback Model, the new uniform designs consistently resulted in lower Mean Squared Errors (MSE) when used to train Kriging metamodels. For example, in the Wood Model, Unew(9^4) designs generally showed lower MSEs than U9(9^4) designs across different Kriging bases (Table 7).
  • Computational Efficiency: The coordinate zero-gradient (CZG) method is highlighted as being significantly faster than coordinate gradient descent (CGD), achieving results in seconds compared to minutes for the same optimization task.

Limitations and Future Work

  • Computational Cost: While coordinate descent is generally more efficient than gradient descent for high-dimensional problems, constructing UDs for very large numbers of factors and runs can still be computationally intensive.
  • Non-convexity Challenges: The non-convex nature of the CD objective function means that while the TA initialization and coordinate descent aim for good local optima, finding the absolute global optimum is not guaranteed.
  • Future Work: The paper suggests that further research could explore more advanced optimization techniques or hybrid approaches to further improve the efficiency and effectiveness of constructing UDs for continuous domains. The authors also imply that the practical applicability of CDFSS could be further investigated in real-world experimental settings with specific variable constraints.

Summary updated

BibTeX
@inproceedings{lai2024construction,
  author = {Jianfa Lai and Kai-Tai Fang and Xiaoling Peng and Yuxuan Lin},
  title = {{Construction of uniform designs over continuous domain in computer experiments}},
  booktitle = {Commun. Stat. Simul. Comput},
  year = {2024},
  url = {https://openreview.net/forum?id=h1fhlXHw2g},
}
Figure