Physics-enhanced Deep Surrogates Trained End-to-end | Raphaël Pestourie | SciMLCon 2022

Physics-enhanced deep surrogates trained end-to-end | Raphaël Pestourie | SciMLCon 2022

Abstract: We present a physics-enhanced deep-surrogate approach towards developing fast surrogate models for complex physical systems described by PDEs and similar models: We propose a combination of a low-fidelity, explainable physics simulator and a neural network that generates “coarsified” inputs, trained end-to-end to globally match the output of an expensive high-fidelity numerical solver. The low-fidelity model ensures that governing conservation laws and symmetries are respected by design.

Description: In mechanics, optics, thermal transport, fluid dynamics, physical chemistry, climate models, crumpling theory, and many other fields, data-driven surrogate models—such as polynomial fits, radial basis functions, or neural networks—are widely used as an efficient solution to replace repetitive calls to slow numerical solvers. However the reuse benefit of surrogate models comes at a significant cost in training time, where a costly high-fidelity numerical solver must be evaluated many times to provide an adequate training set, and this cost rapidly increases with the number of model parameters (the “curse of dimensionality”). In this proposal, we explore one promising route to increasing training-data efficiency: incorporating some knowledge of the underlying physics into the surrogate by training a generative neural network (NN) “end-to-end” with an approximate physics model. We call this hybrid system a “physics-enhanced deep surrogate” (PEDS). We demonstrate multiple-order-of-magnitude improvements in sample and time complexity on a test problem involving optical metamaterials—composite materials whose properties are designed via microstructured geometries. In inverse design of metamaterials, similar geometric components may be re-used thousands or millions of times in a large structure such as an optical metasurface, making surrogate models especially attractive to accelerate computational design.

In particular, we present a PEDS architecture for modeling transmission through a microstructured multilayer “metasurface”, where the high-fidelity model solves Maxwell’s equations, in which a deep NN is combined with an fast approximate Maxwell solver based on an extremely coarse discretization. In this way, the NN learns to nonlinearly correct for the errors in the coarse model, but at the same time the coarse model “builds in” some knowledge of the physics and geometry. In particular, the coarse model automatically respects conservation of energy and reciprocity, and we can also enforce geometric symmetries, which augments the "trustworthiness" of the model. We compare the result of our PEDS model against a NN-only baseline model as well as previous “space-mapping” (SM) approach where we combine a coarse Maxwell solver with a NN transforming only a low-dimensional parameterization of the fine geometry to a similar low-dimensional parameterization of the coarse geometry. We find that PEDS not only lowers the error of the surrogate for a given amount of data, but it actually seems to improve the asymptotic rate of learning (~5x larger power law), so that the benefits increase as accuracy tolerance is lowered. For 3.5% accuracy, PEDS requires several orders of magnitude less data than the competing approaches. We show through an ablation study that adding information from the downsampled structure increases the accuracy by 15% in a low-data regime. Furthermore, we find that PEDS gains significant additional benefits by combining it with active-learning techniques, and in fact the benefits of active learning seem to be even greater for PEDS than for competing approaches. Although the resulting PEDS surrogate is more expensive to evaluate than a NN by itself, due to the coarse Maxwell solver, it is still much faster than the high-fidelity Maxwell solver (100x in 2D, 10000x in 3D). Furthermore, since our NN generates a coarsified version of the geometry, this output can be further examined to gain insight into the fundamental physical processes affecting the output.
PEDS should not be confused with physics-informed neural networks (PINNs), which solve the full PDE (imposed point wise throughout the domain) for the entire PDE solution (not a surrogate for a finite set of outputs like the complex transmission), and which do not employ any pre-existing solver; indeed, current PINNs tend to be slower than conventional "fine" PDE solvers (e.g., based on finite elements), but offer potentially greater flexibility.