On Properties of Adjoint Systems for Evolutionary PDEs

Brian K. Tran, Ben S. Southworth, Melvin Leok

TL;DR

This paper investigates the geometric structure of adjoint systems for evolutionary partial differential equations (PDEs) across continuous, semi-discrete, and fully discrete levels. It establishes a framework for understanding when discretizing and forming adjoints commute, offering a geometric characterization of the discrepancies between "discretize-then-optimize" and "optimize-then-discretize" approaches. This work is crucial for developing robust and accurate numerical methods in optimization and optimal control of PDEs.

What Problem Does This Paper Address?

In optimization and optimal control problems involving evolutionary PDEs, adjoint systems are essential for efficiently computing gradients. A key question is how discretizing these systems (both in space and time) interacts with the process of forming the adjoint. Specifically, do "discretize-then-optimize" (DtO) and "optimize-then-discretize" (OtD) methods yield the same results? The paper highlights that these approaches generally do not commute, leading to discrepancies in discrete gradients. This work aims to provide a precise geometric understanding of these discrepancies and identify conditions under which they can be controlled or eliminated, particularly by preserving the underlying Hamiltonian structure of the adjoint systems.

Methods

The paper leverages tools from symplectic geometry to analyze adjoint systems. It first establishes an infinite-dimensional Hamiltonian structure for continuous adjoint systems associated with evolutionary PDEs. This geometric framework is then extended to semi-discrete (spatial discretization) and fully discrete (spatial and temporal discretization) levels. A core technique is the characterization of commuting discretizations and adjoint formation through adjoint-variational quadratic conservation laws. For semi-discretization, the paper shows that commuting adjoints and discretization requires the discrete adjoint system to preserve the Hamiltonian structure. For time integration, it demonstrates that commuting adjoints and time integration occurs precisely when the numerical method for the adjoint system is the cotangent lift of the method used for the state dynamics. The paper also introduces the concept of variationally equivariant one-step methods, which are crucial for transferring the order of accuracy from the state dynamics to the adjoint system.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Hamiltonian Structure is Key: The infinite-dimensional Hamiltonian structure of adjoint systems provides a unifying geometric perspective across different discretization levels.
  • Commutativity via Conservation Laws: The commutativity of discretizing and forming adjoints is precisely characterized by the preservation of semi-discrete and fully discrete adjoint-variational quadratic conservation laws.
  • Cotangent Lift for Time Integration: For time integration, the "optimize-then-discretize" approach commutes with forming adjoints if and only if the numerical method for the adjoint system is the cotangent lift of the method for the state dynamics. This is a generalization of existing results for symplectic methods.
  • Discrepancy Explained: The paper provides a geometric explanation for why DtO and OtD methods can lead to different discrete gradients: OtD methods that do not use the cotangent lifted integrator fail to satisfy the discrete adjoint-variational quadratic conservation law.
  • Duality Pairing Matters: The choice of duality pairing in semi-discretization and time integration significantly impacts the resulting discrete adjoint system and its properties, including equilibrium preservation.

Key Results

  • The paper establishes that for semi-discretization, the dual semi-discretization of the adjoint system is unique and preserves the adjoint-variational quadratic conservation law when it commutes with adjoining.
  • For time integration, the cotangent lifted one-step method is shown to be the unique method that covers the state dynamics method and satisfies the discrete adjoint-variational quadratic conservation law.
  • The paper demonstrates that variationally equivariant one-step methods allow the order of accuracy of the state dynamics method to be transferred to the adjoint system, ensuring adjoint consistency.
  • A commutative diagram is presented that summarizes the relationships between continuous adjoint systems, semi-discretized systems, and fully discretized systems, highlighting the role of duality pairings and transformations.

Limitations and Future Work

  • General Nonlinearities: While the paper discusses semilinear PDEs, extending the framework to more general nonlinearities might require further investigation of analytic issues like existence and uniqueness.
  • Multistep Methods: The paper focuses on one-step methods for time integration. The behavior of adjoints with multistep methods is noted as a potential area for future research, as they generally do not possess the same symplectic properties.
  • Space-Time Discretization: The authors propose exploring combined space-time discretization procedures for adjoint systems, potentially using multisymplectic geometry, as a future research direction. This could lead to unified, structure-preserving methods for adjoint sensitivity analysis.
  • Computational Implications: The paper touches upon computational implications, particularly regarding the choice of duality pairing and its effect on numerical results with floating-point arithmetic. Further exploration of these practical aspects could be beneficial.

Summary updated

Abstract

We investigate the geometric structure of adjoint systems associated with evolutionary partial differential equations at the fully continuous, semi-discrete, and fully discrete levels and the relations between these levels. We show that the adjoint system associated with an evolutionary partial differential equation has an infinite-dimensional Hamiltonian structure, which is useful for connecting the fully continuous, semi-discrete, and fully discrete levels. We subsequently address the question of discretize-then-optimize versus optimize-then-discrete for both semi-discretization and time integration, by characterizing the commutativity of discretize-then-optimize methods versus optimize-then-discretize methods uniquely in terms of an adjoint-variational quadratic conservation law. For Galerkin semi-discretizations and one-step time integration methods in particular, we explicitly construct these commuting methods by using structure-preserving discretization techniques.
BibTeX
@article{tran2024properties,
  author = {Brian K. Tran and Ben S. Southworth and Melvin Leok},
  title = {{On Properties of Adjoint Systems for Evolutionary PDEs}},
  journal = {J. Nonlinear sci},
  year = {2024},
  doi = {10.1007/s00332-024-10071-1},
}
Figure