Creating APIs For Machine Learning Models with FastAPI

Today we learn how to easily turn machine learning models into usable APIs using FastAPI in Python.

◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾
📚 Programming Books & Merch 📚

💼 Services 💼

🌐 Social Media & Contact 🌐

[omegasigma]

I'm glad you uploaded a video about FastAPI. We prefer it over Flask.
There are 2 topics where we need some help.
1.) Hosting - How to deploy the app so that other can access it via web? And how to manage the cloud infrastructure?
2.) Frontend - There are now plenty of frameworks and libraries. The standard approach is probably JavaScript, HTML and CSS. But I'm wondering what you think about pure Python libraries like Taipy, FastUI and reflex. What do you think is the best approach here? We would highly appreciate your input. Thanks!

Keep up the great work! 💪💪👍👍

[khandoor]

This was excellent, the capabilities this opens up is really powerful. Good job as always.

[timothyelems]

Exactly what I was looking for! Thanks man!

[dipeshsamrawat]

You are making requested videos. Thank you 💯

[TheDigitalSight]

We use FastAPI more than django and flask, can you please create video on langchain and fastapi as well?

[cheukmingau]

In production the async endpoint should not be used. An async function (coroutine) will be executed in the main thread event loop, and like the event loop in JS inside the browser, it can only execute one coroutine at a time. Running the synchronous, cpu intensive `model.predict` inside the async endpoint will make your prediction endpoint frozen and wait for the underlying cpu predicting the images, so the QPS of your handler is at most one.

Better options could be: 1) Using a synchronous function as the inference endpoint, 2) create a threadpoolexecutor outside of the async function, and use `loop.run_in_executor()` with the threadpoolexecutor declared as it will run the model inside the thread, or 3) use poolexecutor similar to option 2. The problem for option 3 is that multiprocesses requires pickling and you might have to tweak your model case by case.

Also, pickling the model and deserialize in the application api server doesn't reveal the identity and method signatures of that model. If you are the only one who train and deploy that might not be a big problem, but in production you might want to use some inferencing frameworks like Onnxruntime which you just serialize your trained model to the preferred format first (onnxruntime has a very small package size compared to other DL libraries which makes the deployment dependency smaller). Lastly, running scikit-learn model in python doesn't utilize the all the cores in your cpu, whereas other packages usually have higher utilization proportion.

I understand that the model in this video is small in size and is a POC, so with the small size running async and pickling is fine. However, for just some even better CV and NLP models (e.g. BERT) it is nearly impossible to adopt the same approach as in this tutorial.

[NomadicBrian]

I've been using fastAPI for a couple of years now. Just starting with AI models. I had planned on calling models with fastAPI. See if I can do that with a ViT model I've been working with.

[smstudio]

Can we see a hosting video off the same

[thelifehackerpro]

why not use model directly instead of pickle?

[Hardy_]

For me it correctly guesses only numbers 4, 6. For the rest it says they're 7 or 5.

[zedcodes]

Why do I have this error: 'module 'PIL.Image' has no attribute 'ANTIALIAS''? @10:41

[DougSteinberg]

Great video! I'm a web developer and new to ML. Do you have the source code for this project in a Github repo? I would really love to try this out locally.

[Lyphnet]

Please ensure that your Discord server remains joinable. Thanks!

[tcgvsocg]

i am not entire sur i understand how that worst but thx a lot for the video