filmov
tv
Fast Inference for Language Models | PyData Yerevan May 2022
Показать описание
NLP research engineer in Unum, Vladimir Orshulevich, will walk us through the next chapter with the “Fast Inference for Language Models” talk.
This will be a remarkable opportunity to:
- discover about speedups for vanilla hugging face models inference using TensorRT, ONNX, and Nvidia NGC Container that is optimized for GPU acceleration,
- be acquainted with how to make NLP models lighter and fast,
- explore batch size, max_length selection,
- find out about GPU distributed model inference.
===
PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.
PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases.
00:00 Welcome!
00:10 Help us add time stamps or captions to this video! See the description for details.
This will be a remarkable opportunity to:
- discover about speedups for vanilla hugging face models inference using TensorRT, ONNX, and Nvidia NGC Container that is optimized for GPU acceleration,
- be acquainted with how to make NLP models lighter and fast,
- explore batch size, max_length selection,
- find out about GPU distributed model inference.
===
PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.
PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases.
00:00 Welcome!
00:10 Help us add time stamps or captions to this video! See the description for details.
0:23:25
Fast Inference for Language Models | PyData Yerevan May 2022
0:11:58
Fast Inference of Mixture-of-Experts Language Models with Offloading
0:01:08
Accelerate Big Model Inference: How Does it Work?
0:10:14
Small Language Model Inference Speed Testing - 14x Faster than PyTorch on a Dell Laptop?
0:05:34
How Large Language Models Work
0:30:25
Exploring the Latency/Throughput & Cost Space for LLM Inference // Timothée Lacroix // CTO Mist...
0:03:54
StreamingLLM - Extend Llama2 to 4 million token & 22x faster inference?
0:32:07
Fast LLM Serving with vLLM and PagedAttention
1:27:41
From Ants to Active Inference with Daniel Ari Friedman and Andrea Hiott Audio Only
0:00:58
Faster LLM Inference NO ACCURACY LOSS
0:03:37
Mixtral 8X7B Crazy Fast Inference Speed
0:14:19
Llama2.mojo🔥: The Fastest Llama2 Inference ever on CPU
0:10:30
All You Need To Know About Running LLMs Locally
0:01:00
How Cerebras AI inference is 20x faster than competitors
0:03:44
Speeding Up Language Models: Fast Inference with Mixture of Experts
0:28:40
Build an API for LLM Inference using Rust: Super Fast on CPU
0:24:02
'I want Llama3 to perform 10x with my private knowledge' - Local Agentic RAG w/ llama3
0:19:46
Quantization vs Pruning vs Distillation: Optimizing NNs for Inference
0:11:53
Go Production: ⚡️ Super FAST LLM (API) Serving with vLLM !!!
0:02:53
Build a Large Language Model AI Chatbot using Retrieval Augmented Generation
0:01:01
Groq has 2x - 4x Faster Large Language Model Speed!? #groq #llm #inference #aideveloper #aiapi
0:14:42
I Ran Advanced LLMs on the Raspberry Pi 5!
0:10:02
Herbie Bradley – EleutherAI – Speeding up inference of LLMs with Triton and FasterTransformer
0:22:28
PowerInfer: 11x Faster than Llama.cpp for LLM Inference 🔥
Комментарии