Residual Networks (ResNet) [Physics Informed Machine Learning]

Показать описание

This video discusses Residual Networks, one of the most popular machine learning architectures that has enabled considerably deeper neural networks through jump/skip connections. This architecture mimics many of the aspects of a numerical integrator.

This video was produced at the University of Washington, and we acknowledge funding support from the Boeing Company

%%% CHAPTERS %%%
00:00 Intro
01:09 Concept: Modeling the Residual
03:26 Building Blocks
05:59 Motivation: Deep Network Signal Loss
07:43 Extending to Classification
09:00 Extending to DiffEqs
10:16 Impact of CVPR and Resnet
12:17 Resnets and Euler Integrators
13:34 Neural ODEs and Improved Integrators
16:07 Outro

Steve Brunton

Рекомендации по теме

Комментарии

Thank you very much for your videos. I am glad that besides the classical sources to promote science (such as books and papers), your lectures can also be found on youtube. In my opinion, Prof. Bruton is the best provider of youtube lectures and I don't want to miss any of the lectures.

mostafasayahkarajy

Thank you for this content!
Love your approach. Please never stop educating people. We all need teachers like you!:) ps Enjoying reading your book

goodlack

Excellent explanation! For those interested in LLMs residual connections are also featured in the vanilla transformer block. The idea is similar to CNN ResNets, but instead of gradually adding pixel resolution each block adds semantic "resolution" to the original embedded text input.

culturemanoftheages

Many thanks for the extremely good videos. Really well explained and easy to understand.
A video on "Augmented neural ODEs" would go well with "neural ODEs" 😊

lorisdemuth

Hi Steve - this is Vivek Karmarkar! Thanks for the video - great content as usual and keeps me motivated to create my own PINN content as well. Looking forward to the next video in the series and would love to talk PINN content creation with you!
I have been thinking about presenting PINNs with ODEs as examples and its nice to contrast it with Neural ODEs - nomenclature aside, it looks like the power of the NN as universal approximators allows us to model either the flow field (Neural ODEs) or the physical field of interest (PINNs) for analysis which is pretty cool!

physicsanimated

Thank you SO MUCH prof. Steve!! I intend to study neural networks in my graduate courses thanks to your lectures!! :D

saraiva

Thank you for making this. I watched your video on Neural ODEs before I watched this. It is much easier to understand the Neural ODE video now that I have watched this.

I would love to watch a video about the ResNet classifier idea you discuss here. If you have already done that please add a link here.

Thanks, and this is awesome!

sainissunil

You have an impressive ability to simplify complex subjects

Daniboy

ResNet is literally the best thing happened in Deep Learning.

ultrasound

Why are Nueral ODEs not more popular? I’ve never heard of them but I see unet and resnet everywhere

PaulFidika

Does anyone know if transformers have superseded resnets for image classification?

Onewonton

tbh understanding his videos is very difficult, IMO he explains badly. Like 14:14 is the first more complicated part and I don't really get what it is about. I wouldn't understand ResNet from his explanation either if I had no prior knowledge about it. He just assumes that I am some expert in math and DLs

HansPeter-gxew

STEVE MAKE TWO. SMALLER HIGHER LIKE ARRAY ONE DIRECTION OR SYMMETRY LIKE MIRROR. FEEDBACK AND THIS 150.000ageSCIENCE.

cieciurka

Bound, border, infinity, noninfinity, natural, where is the end?! calc machine how it works, integer, costs money costs profits cons in mathematics, NOMIA! ECO? algorithm accuracy, fuzzy logic, integer 0-I. ONE BOOK NO INDIVIDUAL HERE 🎉WHEN YOU SMOOTHING GRADIENT YOU LOSING

cieciurka

IMHO you should speak a little faster and make less jests; for scientists watching this it wastes a time and attention.

maksymriabov

I am a video editor. If you need any help related to video editing you can contact me. I will share my portfolio

suhailea

Residual Networks (ResNet) [Physics Informed Machine Learning]

Residual Networks (ResNet) [Physics Informed Machine Learning]

Neural ODEs (NODEs) [Physics Informed Machine Learning]

A Hands-on Introduction to Physics-informed Machine Learning

AI/ML+Physics Part 3: Designing an Architecture [Physics Informed Machine Learning]

AI/ML+Physics Part 1: Choosing what to model [Physics Informed Machine Learning]

DataLearning: Physics-Informed Neural Networks in Medicine

Hamed Saidaoui, Deep Nurbs : Physics-Informed Neural Networks for Complex Geometries

Janaki Vamaraju: FResNet++: Augmenting Deep Residual Surrogates with Fourier Neural Operators...

ETH Zürich DLSC: Introduction to Deep Learning Part 1

A. Subramaniam - Leveraging physics information in neural networks for fluid flow problems

Neural Differential Equations

Robust learning with implicit residual networks

DeepOnet: Learning nonlinear operators based on the universal approximation theorem of operators.

DDPS | Physics-Informed Learning for Nonlinear Dynamical Systems

Continuous-in-Depth Neural Networks

DataLearning: Physics-Informed Deep Learning - Learning from Small Data

SoK: How Robust is Image Classification Deep Neural Network Watermarking?

What are Autoencoders?

Introduction to Week 9 - From Linear Models to Neural Networks

MSML2020 Invited Talk by Prof. George Karniadakis, Brown University

Physics Informed Attention Temporal Convolutional Network for EEG Based Motor Imagery Classification

How to use backpropagation to create a neural network out of any physical system - Peter McMahon

DeepXDE: A Deep Learning Library for Solving Differential Equations by Lu Lu

Machine Learning for Computational Fluid Dynamics