Local Polynomial Estimation for Sensitivity Analysis on Models With Correlated Inputs

Sébastien Da Veiga, François Wahl, Fabrice Gamboa

Introduction

Global Sensitivity Analysis (GSA) is a critical tool in scientific and engineering modeling, used to determine which input parameters most significantly influence the variance of a model's output. The most common metric for this is the variance-based sensitivity index ($S_i$). However, traditional GSA methods, such as the Sobol’ method or Fourier Amplitude Sensitivity Test (FAST), rely on the assumption that input factors are independent.

In real-world applications—such as chemical kinetics or environmental modeling—parameters are frequently correlated (e.g., temperature and pressure). Ignoring these correlations leads to misleading sensitivity indices, potentially causing researchers to focus on the wrong variables. This paper proposes a computationally efficient, "black-box" approach using local polynomial estimation to perform GSA on models with correlated inputs.

The Challenge of Correlated Inputs

When inputs are correlated, the contribution of a single variable to the output variance is no longer easily separable. Existing workarounds are often either prohibitively expensive, requiring millions of model evaluations, or mathematically complex, involving multidimensional integration of joint probability densities.

Bayesian methods (like Kriging) offer a solution but suffer from the "curse of dimensionality" as the number of inputs increases. The authors seek a middle ground: a method that is statistically rigorous yet practical for engineering workflows.

Methodology: Local Polynomial Regression

The core of the proposed method is the use of Local Polynomial Regression to estimate the conditional moments required for sensitivity indices. Specifically, the first-order sensitivity index is defined as:

$$S_i = \frac{Var(E(Y|X_i))}{Var(Y)}$$

To estimate the numerator, the authors introduce two original estimators:

  1. $\hat{T}_1$: Estimates the conditional expectation $m(x) = E(Y|X=x)$.
  2. $\hat{T}_2$: Estimates the conditional variance $\sigma^2(x) = Var(Y|X=x)$.

Unlike global models, local polynomial estimation fits low-order polynomials (typically linear) to data points within a small neighborhood of a target point. This allows the method to "learn" the relationship between a specific input $X_i$ and the output $Y$ without knowing the model's internal structure.

The Two-Sample Workflow

To maximize efficiency, the authors employ a two-sample approach:

  • Step 1: Run the expensive model on a moderate sample to obtain input-output pairs.
  • Step 2: Use local polynomial smoothing to approximate the relationship between inputs and outputs.
  • Step 3: Use a second, larger sample of inputs only (which are computationally cheap to generate) to calculate the final sensitivity index. This provides high precision without requiring thousands of additional model runs.

Experimental Evaluation

The method was tested against three distinct cases to prove its robustness and accuracy.

1. Analytical Linear Model

A three-factor additive model ($Y = X_1 + X_2 + X_3$) with correlated inputs served as the ground truth. The local polynomial estimators closely matched theoretical values even with small sample sizes ($n=50$).

Sensitivity indices for a three-factor model with correlated inputs, showing the distribution of estimates for each factor.

2. Complex 2D Function

To test the method against non-linearities, the authors used a complex function characterized by sharp peaks and valleys. This case compared the proposed method against Bayesian Kriging approaches, demonstrating that local polynomial estimation can handle highly non-linear surfaces effectively.

Surface plot of the complex 2D function used to test the robustness of the local polynomial estimator against sharp non-linearities.

3. Chemical Kinetics Case Study

The method was applied to a real-world model of the Isomerization of Normal Butane, involving 8 correlated parameters. The analysis successfully identified how parameter importance shifts under different operating conditions, providing actionable insights for chemical engineers.

Key Insights and Results

  • Accuracy and Efficiency: The local polynomial estimators ($\hat{S}^{(1)}$ and $\hat{S}^{(2)}$) provided results comparable to complex Bayesian methods but with significantly simpler implementation.
  • Avoiding Dimensionality Issues: By approximating conditional expectations marginally (one input at a time), the method bypasses the high-dimensional integration required by previous techniques.
  • Flexibility: As a nonparametric, black-box approach, it is highly portable across different scientific fields.

Conclusion

This research provides a practical framework for sensitivity analysis when input independence cannot be assumed. While choosing the correct "bandwidth" (neighborhood size) for smoothing remains a critical step, the use of automated selectors like EBBS makes the method accessible. Future work will look into extending this approach to high-order interactions and multi-output models.

Summary updated

Abstract

Sensitivity indexes when the inputs of a model are not independent are derived from local polynomial techniques. Two original estimators based on local polynomial smoothers are proposed. Both have ...
BibTeX
@article{veiga2009local,
  author = {Sébastien Da Veiga and François Wahl and Fabrice Gamboa},
  title = {{Local Polynomial Estimation for Sensitivity Analysis on Models With Correlated Inputs}},
  journal = {Technometrics},
  year = {2009},
  doi = {10.1198/tech.2009.08124},
}
Figure