Coupled Input-Output Dimension Reduction: Application to Goal-oriented Bayesian Experimental Design and Global Sensitivity Analysis

Qiao Chen, Elise Arnaud, Ricardo Baptista, Olivier Zahm

High-dimensional models are fundamental to modern engineering and physics, yet they are frequently hampered by the "curse of dimensionality." As the number of variables increases, the computational cost of analyzing these systems—whether for optimal sensor placement or identifying key driving parameters—grows exponentially. Traditional approaches often resort to expensive simulations or NP-hard combinatorial searches that are computationally prohibitive for complex models.

In the paper "Coupled Input-Output Dimension Reduction," Chen et al. propose a novel framework to address these challenges. By treating input and output reduction as an interdependent problem and utilizing gradient-based bounds, the authors transform a computationally daunting task into a manageable linear algebra problem.

The Challenge of High-Dimensional Coupling

In fields such as climate modeling or structural engineering, researchers work with functions $G(X) = Y$, where both the input space $X$ (parameters) and the output space $Y$ (states) are high-dimensional. Conventional dimensionality reduction methods typically focus on one side of the equation: Principal Component Analysis (PCA) is often used to reduce the output space, while techniques like Active Subspaces are applied to the inputs.

However, these spaces are intrinsically coupled. The choice of a specific input subspace dictates which output features are relevant, and conversely, focusing on specific outputs highlights different critical inputs. Ignoring this interplay leads to suboptimal reduced-order models that either retain unnecessary complexity or discard vital information. Furthermore, "goal-oriented" tasks—such as selecting the 10 most informative sensor locations from a pool of 1,000—usually require evaluating metrics like Expected Information Gain (EIG), which are notoriously expensive to compute and optimize.

A Gradient-Based Framework for Coupled Reduction

The authors' core innovation is a coupled dimension reduction framework that minimizes the $L^2$-approximation error between the full high-dimensional model and a reduced-order surrogate.

Bypassing Combinatorial Optimization

To avoid the direct calculation of expensive objectives like EIG or Sobol’ indices, the method leverages the Poincaré and Cramér-Rao inequalities. These mathematical tools allow the authors to derive upper and lower bounds for the approximation error. These bounds are defined by two "diagnostic matrices":

  1. $H_X(V_s)$: Quantifies how variations in the input affect a specific output subspace.
  2. $H_Y(U_r)$: Identifies which output components are most informed by a specific input subspace.

The Alternating Eigendecomposition Algorithm

To identify the optimal subspaces, the authors employ an efficient alternating eigendecomposition scheme:

  1. Fix the output subspace: Determine the optimal input subspace by calculating the dominant eigenvectors of $H_X$.
  2. Fix the input subspace: Update the output subspace using the eigenvectors of $H_Y$.
  3. Iterate: Repeat the process until convergence.

For applications like sensor placement or parameter selection, the most informative components are identified simply as the largest diagonal entries of these diagnostic matrices. This approach effectively replaces a "needle-in-a-haystack" combinatorial search with a straightforward sorting task.

Empirical Validation

The framework was validated using two distinct benchmarks in applied mathematics:

  • Conditioned Diffusion: A 1D particle moving in a double-well potential. This test demonstrated the method's ability to handle stochasticity and time-causality, correctly identifying that future observations cannot inform past driving forces—a nuance often missed by decoupled methods.
  • Burgers’ Equation: A non-linear partial differential equation used in fluid dynamics. The authors tested goal-oriented sensor placement, selecting 10 sensors from 100 candidate locations to infer initial conditions. The proposed method outperformed standard PCA-based selection and matched the performance of more complex benchmarks like the Fedorov-Wynn algorithm.

Key Insights and Results

  • Coupling is Essential: The research shows that PCA-based reduction, which ignores the input-output link, often fails to capture the underlying physics of the problem. The coupled approach naturally discovers these relationships.
  • Efficiency and Robustness: The alternating algorithm is remarkably stable, typically converging in approximately three iterations regardless of the starting point.
  • Tight Bounds: The derivative-based bounds were shown to track the true values of EIG and Total Sobol’ indices closely, providing a reliable and inexpensive proxy for optimization.
  • Computational Speedup: By requiring only the model's Jacobian (gradients), the method circumvents the need for the thousands of Monte Carlo simulations usually required for Bayesian experimental design.

Limitations and Future Directions

While powerful, the method relies on the availability of the model's Jacobian. While adjoint methods make this feasible for many PDEs, it may be a limitation for "black-box" models where gradients are noisy or unavailable. Additionally, the current framework focuses on linear projections; extending this to non-linear manifolds using neural networks or non-linear transformations represents a promising avenue for future research. Finally, while the method finds the best subspaces for a given dimension, determining the optimal rank for those subspaces remains an open question.

Summary updated

Abstract

We introduce a new method to jointly reduce the dimension of the input and output space of a function between high-dimensional spaces. Choosing a reduced input subspace influences which output subspace is relevant and vice versa. Conventional methods focus on reducing either the input or output space, even though both are often reduced simultaneously in practice. Our coupled approach naturally supports goal-oriented dimension reduction, where either an input or output quantity of interest is prescribed. We consider, in particular, goal-oriented sensor placement and goal-oriented sensitivity analysis, which can be viewed as dimension reduction where the most important output or, respectively, input components are chosen. Both applications present difficult combinatorial optimization problems with expensive objectives such as the expected information gain and Sobol' indices. By optimizing gradient-based bounds, we can determine the most informative sensors and most influential parameters as the largest diagonal entries of some diagnostic matrices, thus bypassing the combinatorial optimization and objective evaluation.
BibTeX
@article{chen2025coupled,
  author = {Qiao Chen and Elise Arnaud and Ricardo Baptista and Olivier Zahm},
  title = {{Coupled Input-Output Dimension Reduction: Application to Goal-oriented Bayesian Experimental Design and Global Sensitivity Analysis}},
  journal = {SIAM J. Sci. Comput.},
  year = {2025},
  doi = {10.1137/24m1670421},
}
Figure