How to Run PyTorch Models in the Browser With ONNX.js

Run PyTorch models in the browser with JavaScript by first converting your PyTorch model into the ONNX format and then loading that ONNX model into your website or app using ONNX.js. In this video, I take you through this process by building a handwritten digit recognizer that runs in the browser.

Live demo:

The demo sandbox code:

The GitHub repo:

I opened these issues for the bugs mentioned in the video if you want to track their status:

The benefits of running a model in the browser:
• Faster inference times with smaller models.
• Easy to host and scale (only static files).
• Offline support.
• User privacy (can keep the data on the device).

The benefits of using a backend server:
• Faster load times (don't have to download the model).
• Faster and consistent inference times with larger models (can take advantage of GPUs or other accelerators).
• Model privacy (don't have to share your model if you want to keep it private).

Join our Discord community:

Connect with me:

🎵 Kazukii - Return

[fangdahan]

very useful tutorial with potential bug explainations, thank you!

[MaxMitchell]

I'm shocked that this video only has 5k views. Truly one of the better CS tutorials I've seen, especially considering it's a topic as dense as ML. It's also funny as I just had a an assignment on the MNIST dataset and was going to make a video trying to add it to my website, but it looks like you beat me to it :)

[santoshmohanram]

This is the only video that explained me pytorch to onnx export and deployment on web clearly. Thanks for your contribution brother. Big fan of your work. Please do make more contents like this brother. May God bless you❤.

[pratikhmanas]

Really Interesting. Thank you for sharing :)

[krivar]

What a great tutorial. Clear, concise and handles some common pitfalls. Plus yo ulook like Matt Damon. Awesome :D

[momonga.]

Wow this is great. I would love to see running a pytorch model in Android video because there's not a lot on that out there atm.

[thor]

watched first 30 seconds, said f*ckin cool...good sign...ill continue to watch now :)

[jothamsoboyejo]

Great video bro, Could you make a video on quantization and deploying large models to the web

[Joseph]

Great Tutorial, would be awesome if you can do somenting similar with image regognition :D

[tylersnard]

Thank you for posting this! Can you do a tutorial on using onnx.js for real-time object detection in the browser?

[BalaSakthi]

Hi! Thanks for your nice tutorial, Is it possible to save the ONNX model in the webbrowser indexeddb while each time we refresh the page or open it again we no need to wait for a model to be downloaded.

[coltvideolibrary]

Thank you for making this video, it is very helpful for somebody new. I am trying to load my image segmentation model created using fastAi. I ran into the problem where the data was not structured properly for my model. You fixed it by modifying your forward function to repackage the data. Currently I do not have a forward function in my code and am trying to learn how to add one. If I did not have one when I trained my model the first time, will I have to retrain it? I wonder if I can simply tack on a forward function in the onnx conversion script where I load my pretrained model from pkl file.

[TimothyGraupmann]

That was an epic video! I would have fixed my canvas input to format the data into the expected Python format rather than change the model. That's just personal preference though. Has Onnx.js added training or does it still just do interference mode?

[anphanpro]

this is such an awesome tutorial! Hope you keep great work. :D is app not working?

[TonyStark-lvkv]

Can we use this technique in chrome extensions that make use of Pytorch models

[anshumantekriwal]

how to accept images of different shapes?
Usually we write a function to convert all the shapes into one particular shape accepted by the model;
what should we do here?

[danieladrose]

Thanks for that! But I have an error: opening the html-file I only get a blank browser. It´s the same in Firefox, Chrome or Edge. Do you know why?

[UniverseGames]

I have one problem while running a py script it's say, "Using Backend: pytorch" why this, any help

[jaden]

Since react native is a JS framework, does that mean ONNX.js can work in a react native app?

[elaith]

When trying to load onnx model I'm getting "Uncaught (in promise) Error: graph attribute is not supported yet". Anyone had this problem yet?