filmov
tv
Accelerate Transformer inference on CPU with Optimum and ONNX
Показать описание
In this video, I show you how to accelerate Transformer inference with Optimum, an open source library by Hugging Face, and ONNX.
I start from a DistilBERT model fine-tuned for text classification, export it to ONNX format, then optimize it, and finally quantize it. Running benchmarks on an AWS c6i instance (Intel Ice Lake architecture), we speed up the original model more than 2.5x and divide its size by two, with just a few lines of simple Python code and without any accuracy drop!
⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos ⭐️⭐️⭐️
I start from a DistilBERT model fine-tuned for text classification, export it to ONNX format, then optimize it, and finally quantize it. Running benchmarks on an AWS c6i instance (Intel Ice Lake architecture), we speed up the original model more than 2.5x and divide its size by two, with just a few lines of simple Python code and without any accuracy drop!
⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos ⭐️⭐️⭐️
Accelerate Transformer inference on CPU with Optimum and Intel OpenVINO
Accelerate Transformer inference on CPU with Optimum and ONNX
Accelerate Transformer inference on GPU with Optimum and Better Transformer
Accelerate Big Model Inference: How Does it Work?
Accelerating Stable Diffusion Inference on Intel CPUs with Hugging Face (part 1) 🚀 🚀 🚀
Accelerate Transformer inference with AWS Inferentia
Hugging Face + OpenVINO™ | Intel Software
Scaling inference on CPUs with TorchServe
Fast T5 transformer model CPU inference with ONNX conversion and quantization
Accelerate Transformer inference with AWS Inferentia 2
Supercharge your PyTorch training loop with Accelerate
Accelerate Transformer Model Training with Hugging Face and Habana Labs
Google Just Turned the RPi into a Supercomputer...
'Accelerating transformers for inference of giant NLP models at scale', Sudeep Bhoja, d-Ma...
PyTorch 2.0 Q&A: Optimizing Transformers for Inference
Accelerate PyTorch Transformers with Intel Sapphire Rapids, part 2
Nvidia CUDA in 100 Seconds
Weights & Biases Webinar: Accelerating Diffusion with Hugging Face
How to Sparsify BERT for Better CPU Performance & Smaller File Size
Pipeline parallel inference with Hugging Face Accelerate
Running a Hugging Face LLM on your laptop
Accelerating Stable Diffusion Inference on Intel CPUs with Hugging Face (part 2) 🚀 🚀 🚀
How to run Large AI Models from Hugging Face on Single GPU without OOM
Supercharge your PyTorch training loop with 🤗 Accelerate
Комментарии