NVIDIA TensorRT 8 Released Today: High Performance Deep Neural Network Inference

NVIDIA TensorRT allows high-performance inference of TensorFlow and PyTorch neural networks. Today NVIDIA released version 8 of this amazing framework. The new version includes sparsity, which optimizations to prune weak connections that do not contribute to the overall calculation of the network. TensorRT 8 allows transformer optimizations and BERT-Large achievement.

Getting Started with TensorRT 8:

Installation Instructions for Windows, Linux and Cloud:

0:44 Quantized Network (QAT)
0:55 Sparsity
1:46 Setup TensorRT
3:10 Using the TensorRT 8 Jupyter 8 Notebook
6:21 Query BERT
6:54 Ask BERT your Own Question
8:00 BERT Weaknesses

* Follow Me on Social Media!

[patrikpatrik]

Thanks for a very interesting video, I would love to see a video about using tenstorRT for object detection in images

[rishinigam]

@Jeff Really great stuff would really like to see implement a GAN like StackGAN on a new or custom dataset like PITT Ads data CVPR 2017

[BioDynamicIndustries]

Hey Jeff, hope you see this. Can you speak about tensorflow & AMD Navi cards? Will we ever be able to work with these cards?

[kaushiksb]

How much boost does the extra RGB give ? (jk)