Physics-Informed Neural Networks in Julia

Показать описание

-------

-------

-------

Timestamps:
00:00 Introduction
00:24 What is a PINN?
00:54 Interpretation of the Poisson problem
01:32 Informing neural network of the physics
03:14 Problem with automatic differentiation
04:15 Manual differentiation of a shallow MLP
07:29 Batched Execution of the neural network
08:36 Imports
08:59 Constants
09:59 Forcing Function & Analytical Solution
10:30 Setting the random seed
10:41 Sigmoid activation function
10:52 Initialize weights & bias of the neural network
13:51 Forward/Primal pass of the network
15:02 Plot initial prediction & analytical solution
18:31 Manual input-output differentiation
23:36 Check correctness with automatic differentiation
26:13 Randomly draw collocation points
28:30 Implement forward loss function
33:11 Testing the outer autodiff call
35:49 Training loop
38:34 Loss plot
39:09 Final PINN prediction
40:03 Summary
42:35 Outro

Machine Learning & Simulation

Рекомендации по теме

Комментарии

With regards to boundary conditions (BCs) ; whether Dirichlet or Neumann you can always do a function composition that a priory satisfies the BCs and have a way much faster PINNS training.

nicklam

Very nice video. Truly showing the potential of Julia for sciml! I’m curious have you compared this Julia algorithm with Jax? It seems like much faster than training in Jax. However, I’m also worried about what if I need to construct mlp rather than one layer net which is most common situation in ml? How about high dimensional data rather than 1d data? Does that also increase the complexity to use Julia?

ziweiyang

Very nice presentation of the basic ideas of PINNs.

It seems to me that this is exactly the same as the Weighted Residual Method, using "collocation weights" from the 1970s and 1980s (possibly earlier?), where the "neural network" (your 1-hidden-layer "ansatz") is what is there denoted the "trial solution". I guess this is named "spectral methods" in some fields...

In the Weighted Residual Method, if one uses a trial solution which is a parameter linear combination of finite support basis functions, the result is a Finite Element Method. One variant is the Galerkin method.

For dynamic systems, a common approach is to specify trial solutions in the spatial directions, and then let the weights be time functions. The WRM then leads to ordinary differential equations in time for the weights.

Question: how are PINNs used for dynamic systems? Do you do the same as in WRM? E.g., using Forward NNs as "trial solutions" in space, and use time varying weights?

berntlie

Great video. Does your network have to make use of diff. eq. for it to fit the definition of a PINN? If I embed physics into my NN in some other way (so not using diff. eq.) can it still be called a PINN?

pietheijn-vogt

Hi Felix K. You mention that Julia does not support multiple layers of autodiff. I am curious because I am starting up a PINN project in science context and I am in doubt which framework to choose (I am working with 2D and potentially 3D fluid flow (slow fluids).
I wonder if you have a recommendation for which framework (Tensorflow, JAX, Julia etc.) is more suited for PINN work?

ksiuxjd

Hi, thanks for the video. I am kind of confuse, why do you define manually the differentiation if you can use the Zygote.gradient?

josep

Could you please provide an example of how to solve heat diffusion eq 1D with a heterogeneous domain?

ahmedshakiraliali

Is there a reason why use Jupyter on Pluto?

Am-peiy

Physics-Informed Neural Networks in Julia

Physics-Informed Neural Networks in Julia

Decomposition Physics Informed Neural Networks | Kirill Zubov | JuliaCon2021

JuliaCon 2020 | Julia for PDEs - Physics Informed Neural Networks for Automated PDE.. | Kirill Zubov

Automated Solution of Symbolic Partial Differential Equations via Physics-Informed Neural Networks

Introduction to Scientific Machine Learning 2: Physics-Informed Neural Networks

Scientific Machine Learning: Physics-Informed Neural Networks with Craig Gin

Teaching Neural Network to Solve Navier-Stokes Equations

What Are Physics Informed Neural Networks (PINNs) ?

Physics Informed Machine Learning: High Level Overview of AI and ML in Science and Engineering

Physics-Informed Neural Networks (PINNs) - Application Use Cases

Introduction to Physics Informed Neural Networks | A hands on, project based course

Physics-Informed Neural Networks | Misconceptions

Physics-Informed ML Simulator for Wildfire Propagation | F Calisto, S Azeglia et al | JuliaCon2021

Mixing Differential Equations and Neural Networks for Physics-Informed Learning

The Julia SciML Ecosystem: Scientific Machine Learning as a Software Problem - Chris Rackauckas

AI/ML+Physics Part 1: Choosing what to model [Physics Informed Machine Learning]

DataLearning: Physics-Informed Neural Networks in Medicine

Physics Informed Neural Networks

PINNs for Optimization and Control

Physics Informed Neural Networks -- Introduction

Julia and NeuralPDE

Physics-Informed Neural Networks in JAX (with Equinox & Optax)

Solving Partial Differential Equations using Physics-Informed Neural Networks (PINN)

Modeling Plasma Physics with NeuralPDE.jl | Gabriel Birnbaum | SciMLCon 2022