Rank adaptive tensor recovery based model reduction for partial differential equations with high-dimensional random inputs

Kejun Tang, Qifeng Liao

TL;DR

This paper introduces a novel "Rank Adaptive Tensor Recovery" (RATR) approach to efficiently model complex physical systems described by partial differential equations (PDEs) with many random parameters. By combining dimensionality reduction techniques with a new tensor recovery method that automatically determines the optimal "rank" of the model, they achieve accurate predictions with significantly reduced computational cost.

What Problem Does This Paper Address?

Many real-world physical and engineering problems are modeled by PDEs that involve uncertainties, often stemming from numerous random input parameters (e.g., material properties, environmental conditions). Simulating these systems accurately requires solving the PDE for many different combinations of these random inputs, which quickly becomes computationally prohibitive due to the "curse of dimensionality." This is especially true when the outputs of interest are complex spatial fields (like temperature or velocity distributions) that also have high dimensionality. Existing methods like stochastic spectral methods struggle with high-dimensional inputs, and while techniques like sparse polynomial chaos expansions and stochastic collocation offer improvements, they often require prior knowledge of model "ranks" or can be computationally expensive. This paper aims to tackle the challenge of high-dimensional inputs and outputs in PDE models by developing a more efficient and adaptive surrogate modeling approach.

Methods

The core of this work lies in a two-pronged approach:

  • Manifold Learning for Output Reduction: They use Kernel Principal Component Analysis (kPCA) to reduce the dimensionality of the high-dimensional output fields. kPCA is effective at capturing non-linear structures in the data, providing a more compact representation of the system's behavior.
  • Rank Adaptive Tensor Recovery (RATR) for Input Uncertainty: For each reduced dimension (kPCA mode), they employ a novel Rank Adaptive Tensor Recovery (RATR) algorithm. This algorithm is designed to compute the coefficients of a sparse polynomial chaos expansion (a common surrogate modeling technique) by recovering "data tensors." Crucially, RATR does not require prior knowledge of the tensor's rank, automatically adapting to find the optimal rank for accurate recovery. This is achieved through a rank-one update procedure and a non-convex optimization approach with novel initialization strategies.

Datasets

The paper does not introduce new benchmark datasets. Instead, it uses numerical simulations of well-defined PDE problems to generate data for evaluation. These include:

  • Diffusion Problems: Two test cases involving diffusion equations with random coefficients, varying correlation lengths, and different grid sizes.
  • Stokes Problem: A test case for incompressible flow, also with uncertain viscosity.

The data for these problems is generated by solving the deterministic versions of the PDEs using high-fidelity numerical schemes (finite element methods) for various realizations of the random input parameters.

Key Insights and Ideas

  • Automatic Rank Determination is Crucial: The paper highlights that for complex systems, the "rank" of the underlying tensor representation is often unknown a priori. Their RATR method's ability to automatically discover this rank is a significant advantage over existing fixed-rank methods.
  • kPCA as a Powerful Preprocessing Step: Using kPCA to reduce the dimensionality of the high-dimensional output fields before applying tensor recovery is shown to be highly effective, allowing the subsequent tensor methods to focus on the essential variations.
  • Initialization Matters for Non-Convex Optimization: The paper emphasizes the importance of good initialization strategies for the non-convex optimization problems inherent in tensor recovery. Their proposed rank-one update procedure provides a stable and efficient way to initialize the process.
  • Sparsity is Key: The success of their approach relies on the inherent sparsity of the polynomial chaos coefficients, which is confirmed by their numerical results.

Key Results

The RATR-collocation approach demonstrated significant improvements in accuracy and efficiency compared to existing methods:

  • Reduced Validation Errors: In numerical experiments (diffusion and Stokes problems), RATR consistently achieved smaller validation errors compared to standard fixed-rank tensor recovery methods, especially when using optimal initializations.
  • Efficient Rank Adaptation: The RATR algorithm efficiently identified small estimated CP ranks for the data tensors, indicating that the underlying low-dimensional structure is well-captured. For instance, in Test Problem 1, the maximum estimated CP rank was four.
  • Computational Cost Savings: The paper argues that RATR-collocation is significantly more computationally efficient than traditional tensor grid or sparse grid collocation methods, particularly for high-dimensional problems, due to its ability to work with a smaller number of observation points.
  • Accurate Predictions: For a given realization of the random input, the RATR-collocation approximation was visually indistinguishable from the high-fidelity finite element solution for both flow streamlines and pressure fields.

Limitations and Future Work

  • High-Rank Problems: The authors acknowledge that the RATR-collocation approach might not be as efficient for problems with very high-rank tensors. They suggest that a possible solution for such cases could involve adaptivity based on the physical properties of the underlying PDE models, such as domain decomposition methods.
  • Stability Analysis: While the paper provides theoretical analysis for the stability of initialization strategies and the rank-one update procedure, the stability analysis for the generalized lasso problem during arbitrary iteration steps of the alternative minimization procedure remains an open problem.
  • Future Work: The authors suggest that designing and analyzing strategies for efficiently recovering high-rank tensors, potentially using domain decomposition methods, will be the focus of their future work.

Summary updated

Abstract

This work proposes a systematic model reduction approach based on rank adaptive tensor recovery for partial differential equation (PDE) models with high-dimensional random parameters. Since the standard outputs of interest of these models are discrete solutions on given physical grids which are high-dimensional, we use kernel principal component analysis to construct stochastic collocation approximations in reduced dimensional spaces of the outputs. To address the issue of high-dimensional random inputs, we develop a new efficient rank adaptive tensor recovery approach to compute the collocation coefficients. Novel efficient initialization strategies for non-convex optimization problems involved in tensor recovery are also developed in this work. We present a general mathematical framework of our overall model reduction approach, analyze its stability, and demonstrate its efficiency with numerical experiments.
BibTeX
@article{tang2020rank,
  author = {Kejun Tang and Qifeng Liao},
  title = {{Rank adaptive tensor recovery based model reduction for partial differential equations with high-dimensional random inputs}},
  journal = {Journal of Computational Physics},
  year = {2020},
  url = {https://openreview.net/forum?id=5XSHWQQN3jK},
}
Figure