Tim Dettmers—k-bit Inference Scaling Laws

Показать описание

Tim is a PhD student at the University of Washington working on representation learning, and hardware optimized deep learning. In this presentation, Tim presents his ICML poster on "k-bit inference scaling laws", or why using 4-bit inference for Large Language Models is optimal for k-bit zero-shot performance.

The Inside View

Рекомендации по теме

Комментарии

Great video, and the research question is fascinating. I am wondering what happens when doing calculations. Okay, the weights are quantized on k bits, but when we compute attention scores or activations, do we keep the calculations in k bits, or is everything dequantized beforehand, and then quantized again?

yacinegaci

know him since childhood, was somekind of weird sometimes :D BR

nishcologne

Tim Dettmers—k-bit Inference Scaling Laws

Tim Dettmers—k-bit Inference Scaling Laws

8-bit Methods for Efficient Deep Learning with Tim Dettmers

Democratizing Foundation Models via k-bit Quantization - Tim Dettmers | Stanford MLSys #82

QLoRA: Efficient Finetuning of Quantized Large Language Models (Tim Dettmers)

QLoRA: Efficient Finetuning of Quantized LLMs | Tim Dettmers

8-bit Methods for Efficient Deep Learning -- Tim Dettmers (University of Washington)

8-bit Optimizers via Block-wise Quantization

Tim Dettmers | QLoRA: Efficient Finetuning of Quantized Large Language Models

AI on your phone? Tim Dettmers on quantization of neural networks — #41

Scaling Laws for Sparsely-Connected Foundation Models

Scaling Laws for Language Transfer Learning | Christina Kim | OpenAI Scholars Demo Day 2021

LoRA explained (and a bit about precision and quantization)

Tim Dettmers: Personal Side of Academia, How to pick your Grad School, RTX 3000 FAQ

Panel discussion #1 | with Tim Dettmers, Johnathan Frankle, Julien Launay and Ce Zhang

Adam Grzywaczewski | The scaling laws of AI Why neural networks continue to grow

The Best GPU for Deep Learning

QLoRA: Quantization for Fine Tuning

Tomek Korbak—Pretraining Language Models with Human Preferences

'On Scaling Laws, Emergent Behaviors, and AI Democratization Efforts' - Predictable AI Eve...

Stanford CS25: V4 I Demystifying Mixtral of Experts

Lecture 05 - Quantization (Part I) | MIT 6.S965

Challenges and Applications of Large Language Models

How to Build a Deep Learning Machine - Everything You Need To Know

Hailey Schoelkopf—Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling