TensorRT for Beginners: A Tutorial on Deep Learning Inference Optimization

preview_player
Показать описание
In this tutorial we are exploring NVIDIA's TensorRT, a deep learning inference optimizer.

We are walking step-by-step through a Jupyter notebook that boosts inference speeds for an object detection model using pyTorch.

This tutorial is intended to be beginner-friendly but requires prior knowledge in Python and familiarity with Neural Networks and Machine Learning.

Jupyter notebook:

Further Reading:

Edit:
In the video I said TensorScript but meant TorchScript.
At the very end I also mention "MAP" without explaining Mean Average precision:

Рекомендации по теме
Комментарии
Автор

Finally someone doing a nice tutorial on it!! I try later to make it work in Touchdesigner. I usually convert the model with the ultralytics.export this actually destroys model capabilities. Let s see if this optimisation is better. Thanks 🙏

unveil
Автор

Hi hi
Thank you for this video. I'm extremely glad I found it because I'm currently working with a Jetson Nano and a Yolov5 model which must be trt optimised.

vicradon
Автор

Hi, if you can help me... I need install in I put it on path ... /bin and .../lib, but what i have to do to use in stable diffusion??? 🙂

heitorvelloso
Автор

good content..I am about to start my PhD in CS and am exploring a few ML research projects and all your 4 videos were really resourcefull...Keep doing these!!

shaunakgalvankar
Автор

Hello, i love this tutorial, it is very easy to learn, but i got some problem when convert some pt to tensort which some model. Yolo easy convert to engine but i try with retinanet and it’s doesnt work ! I dont know the reason why cant convert. I hope u explain for me, thanks

truongnguyennhat
Автор

Quick question : I noticed that the compilation part takes a lot of GPU memory. After compiling, it seems like that GPU memory is not being freed. Have you experienced that before ?
Have you tried compiling before, saving the compiled model and then loading it in inference ? Thanks

valentinfontanger
Автор

hey amazing video can you also do accuracy comparision

racocat
Автор

does it also impact accuracy of the model if you use torch.compile with backend inductor (which is the default)?

Gerald-xgrq
join shbcf.ru