What is Post Training Quantization - GGUF, AWQ, GPTQ - LLM Concepts ( EP - 4 ) #ai #llm #genai #ml

Показать описание

All thanks to -

@MaartenGrootendorst

For his amazing work - In this video, I have used the colab file built by him to explain this concept. Maarten is quite talented, do check out his work!

🎌🎌 Join this channel to get access to perks:

Рекомендации по теме

Комментарии

well expalined.. Lot many concepts got cleared

sqlsql

Hey akhil, could you make this course in such a way that after doing this one can at least apply at your company for an internship.

If not then atleast make a roadmap mentioning all the keywords one can search and learn from the internet. Since i am a full stack I don't have much idea of the ai landscape.

mitejmadan

Hi Akhi, Hoping that you can help, I have an Alienware m18 R2 with an Intel i9-14900HX, NVIDIA RTX 4090 (24GB), 64GB RAM, and 8TB storage. For extra information I don't plan to use this for high intensive tasks like model training or any other such high intensive computing tasks, i only mainly will be using it for analysing my business documents and also writing 20 minute elaborate stories based on a five step story structure. I wanted to use the 70B model to generate the best possible results for these smaller less intensive type tasks. Based on my system specs which which optimisation method would you recommend. GPT Q, GGUF, or AWQ ad would you have any additional advice on the best way to optimise based on my use case requirements?

theuniversityofthemind

very fast explanation. pls be slow from next time. hard to follow.

rahuldebdas

What is Post Training Quantization - GGUF, AWQ, GPTQ - LLM Concepts ( EP - 4 ) #ai #llm #genai #ml

Quantization in deep learning | Deep Learning Tutorial 49 (Tensorflow, Keras & Python)

8.2 Post training Quantization

Quantization explained with PyTorch - Post-Training Quantization, Quantization-Aware Training

Quantization in Deep Learning (LLMs)

Quantization vs Pruning vs Distillation: Optimizing NNs for Inference

What is Post Training Quantization - GGUF, AWQ, GPTQ - LLM Concepts ( EP - 4 ) #ai #llm #genai #ml

Model Quantization in Deep Neural Network (Post Training)

Practical Post Training Quantization of an Onnx Model

MLOps MLFlow: Crafting OpenVino Models with MLflow. #datascience #machinelearning

Post-training Quantization in TensorFlow Lite (TFLite)

Start Post-Training Static Quantization | AI Model Optimization with Intel® Neural Compressor

GPTQ : Post-Training Quantization

AdaRound: Revolutionizing Post Training Quantization

Which Quantization Method is Right for You? (GPTQ vs. GGUF vs. AWQ)

Video #203 GPTQ: Accurate Post-Training Quantization For Generative Pre-Trained Transformers

Post-Training Quantization on Diffusion Models (CVPR 2023)

PD-Quant: Post-Training Quantization based on Prediction Difference Metric [CVPR2023]

Get Started Post-Training Dynamic Quantization | AI Model Optimization with Intel® Neural Compressor...

9.2 Quantization aware Training - Concepts

Recipes for Post-training Quantization of Deep Neural Networks (Abstract)

Lecture 05 - Quantization (Part I) | MIT 6.S965

Deep Dive on PyTorch Quantization - Chris Gottbrath

EE545 (Week 6) More on Quantization and Quantization Aware Training (Part III)

Optimize your models with TF Model Optimization Toolkit (TF Dev Summit '20)