Speeding up Computational Morphogenesis with Online Neural Synthetic Gradients

Yuyu Zhang, Heng Chi, Binghong Chen, Tsz Ling Elaine Tang, Lucia Mirabella, Le Song, Glaucio H. Paulino

Abstract

A wide range of modern science and engineering applications are formulated as optimization problems with a system of partial differential equations (PDEs) as constraints. These PDE-constrained optimization problems are typically solved in a standard discretize-then-optimize approach. In many industry applications that require high-resolution solutions, the discretized constraints can easily have millions or even billions of variables, making it very slow for the standard iterative optimizer to solve the exact gradients. In this work, we propose a general framework to speed up PDE-constrained optimization using online neural synthetic gradients (ONSG) with a novel two-scale optimization scheme. We successfully apply our ONSG framework to computational morphogenesis, a representative and challenging class of PDE-constrained optimization problems. Extensive experiments have demonstrated that our method can significantly speed up computational morphogenesis (also known as topology optimization), and meanwhile maintain the quality of final solution compared to the standard optimizer. On a large-scale 3D optimal design problem with around 1,400,000 design variables, our method achieves up to 7.5x speedup while producing optimized designs with comparable objectives.

Introduction

Designing optimal structural components, such as bridge trusses or aircraft wings, is a complex mathematical challenge known as PDE-constrained optimization or computational morphogenesis. In this field, engineers seek the ideal material distribution to maximize performance (e.g., stiffness) under specific loads. For high-resolution 3D designs involving millions of variables, traditional iterative solvers are prohibitively slow because each step requires solving massive systems of linear equations to obtain exact gradients.

While deep learning (DL) has been explored to accelerate this process, existing "offline" methods often struggle with high data generation costs, poor generalization to new geometries, and a lack of physical accuracy, sometimes resulting in disconnected or "floating" structural members.

Methods: Online Neural Synthetic Gradients (ONSG)

To overcome these limitations, we introduce the Online Neural Synthetic Gradients (ONSG) framework. Unlike traditional DL approaches that rely on pre-training, ONSG utilizes online learning.

Overview of the ONSG framework showing the two-scale optimization scheme and online learning process.

The Two-Scale Optimization Scheme

As shown in the framework overview, the system employs a two-scale strategy to maintain scalability. The design space is partitioned into local blocks (e.g., $N_B \times N_B$ elements). The neural network takes local material density and coarse-grained global physics information as input to predict "synthetic" gradients for that specific block. This local approach ensures that the model's memory footprint remains manageable even as the overall problem size scales to millions of variables.

Online Learning and Switching

The model begins with no prior knowledge. During the initial iterations, exact gradients are calculated using traditional physics solvers to build a training set. Once the neural network achieves sufficient accuracy, the optimizer switches to using synthetic gradients—cheap, predicted updates that bypass the expensive full-physics simulations. Periodic "reality checks" with exact solvers ensure the optimization remains physically grounded.

Results and Performance

We evaluated ONSG on large-scale 3D benchmarks, including Cantilever and MBB beams, with up to 1.5 million design variables.

Comparison of convergence rates (Objective vs. Wall-clock time) for the standard solver and ONSG with various block sizes.

As illustrated in the convergence plots, ONSG significantly reduces the wall-clock time required to reach a converged solution. On a 3D problem with 1.4 million variables, ONSG achieved a 7.5x speedup compared to a state-of-the-art GPU-accelerated solver. Crucially, the final designs produced by ONSG are visually and functionally comparable to those from standard solvers, often achieving similar or even slightly better objective values. The "noise" inherent in synthetic approximations can occasionally help the optimizer escape local minima that trap traditional solvers.

Conclusion

The ONSG framework represents a significant step forward in accelerating structural design. By combining the rigor of PDE-constrained optimization with the efficiency of online deep learning, it enables high-resolution 3D morphogenesis at a fraction of the traditional computational cost. Future work will explore extending this framework to more complex multiphysics problems, such as fluid dynamics and heat transfer.

Paper Citation: Zhang, Y., Chi, H., Chen, B., et al. "Speeding up Computational Morphogenesis with Online Neural Synthetic Gradients." arXiv:2104.12282v2 [cs.AI].

Summary updated

Abstract

A wide range of modern science and engineering applications are formulated as optimization problems with a system of partial differential equations (PDEs) as constraints. These PDE-constrained optimization problems are typically solved in a standard discretize-then-optimize approach. In many industry applications that require high-resolution solutions, the discretized constraints can easily have millions or even billions of variables, making it very slow for the standard iterative optimizer to solve the exact gradients. In this work, we propose a general framework to speed up PDE-constrained optimization using online neural synthetic gradients (ONSG) with a novel two-scale optimization scheme. We successfully apply our ONSG framework to computational morphogenesis, a representative and challenging class of PDE-constrained optimization problems. Extensive experiments have demonstrated that our method can significantly speed up computational morphogenesis (also known as topology optimization), and meanwhile maintain the quality of final solution compared to the standard optimizer. On a large-scale 3D optimal design problem with around 1,400,000 design variables, our method achieves up to 7.5x speedup while producing optimized designs with comparable objectives.
BibTeX
@inproceedings{zhang2021speeding,
  author = {Yuyu Zhang and Heng Chi and Binghong Chen and Tsz Ling Elaine Tang and Lucia Mirabella and Le Song and Glaucio H. Paulino},
  title = {{Speeding up Computational Morphogenesis with Online Neural Synthetic Gradients}},
  booktitle = {IJCNN},
  year = {2021},
  url = {https://openreview.net/forum?id=BMbVcsimZc},
}
Figure