Llama 3.1 405b model is HERE | Hardware requirements

preview_player
Показать описание
In this video, we dive into Meta’s latest AI breakthrough: the Llama 3.1 405B model! Learn about its state-of-the-art capabilities, Inference requirement of over 16K NVIDIA H100 GPUs, supporting a massive 128K context length. Discover how this model excels in general knowledge, multilingual translation, and more, pushing the boundaries of AI technology. Whether you’re an AI enthusiast or a developer, this video covers everything you need to know about the Llama 3.1’s groundbreaking features and applications. Don’t miss out on the future of AI!
  1. TECHNO PREMIUM
  2. llama3.1
  3. llama 405b model
  4. big model
  5. pytorch
  6. lmstudio
Рекомендации по теме
Llama 3.1 405b 0:11:59

Llama 3.1 405b model is HERE | Hardware requirements

Llama 405b BEAST 0:06:12

Llama 405b BEAST already exploited | Here’s how

Zuck's new Llama 0:04:13

Zuck's new Llama is a beast

Llama 3.1 Review 0:17:23

Llama 3.1 Review – A first look at Llama 405B and other updated Llama 3.1 models

Llama 405b: Full 0:26:50

Llama 405b: Full 92 page Analysis, and Uncontaminated SIMPLE Benchmark Results

Meta Llama 3.1 0:00:50

Meta Llama 3.1 is Game Over for GPT 4o ❓

Llama 3.1 is 0:07:05

Llama 3.1 is ACTUALLY really good! (and open source)

Start Running LLaMA 0:03:38

Start Running LLaMA 3.1 405B In 3 Minutes With Ollama

LLaMA 405b Fully 0:10:02

LLaMA 405b Fully Tested - Open-Source WINS!

LLaMA 405b is 0:15:51

LLaMA 405b is here! Open-source is now FRONTIER!

How To Run 0:06:21

How To Run Llama 3.1: 8B, 70B, 405B Models Locally (Guide)

Llama 3.1 405b 0:32:49

Llama 3.1 405b Deep Dive | The Best LLM is now Open Source

LLAMA-3.1 405B: Open 0:13:55

LLAMA-3.1 405B: Open Source AI Is the Path Forward

Llama-3.1 (405B, 70B, 0:05:48

Llama-3.1 (405B, 70B, 8B) + Groq + TogetherAI + OpenWebUI : FREE WAYS to USE ALL Llama-3.1 Models

How to Use 0:02:35

How to Use Meta's LLaMA 3.1 405B Model AI for Free | No Chat Limits, No Deployment Needed

Breaking Down Meta's 0:21:42

Breaking Down Meta's Billion Dollar LLM Blueprint [Llama-3.1 Full Breakdown]

How Did Llama-3 0:13:55

How Did Llama-3 Beat Models x200 Its Size?

'I want Llama3.1 0:25:34

'I want Llama3.1 to perform 10x with my private knowledge' - Self learning Local Llama3.1 ...

Llama 3.1 405b 0:00:53

Llama 3.1 405b & 70b leaked benchmarks #ai #chatgpt #artificialintelligence

Llama-3.1 (Fully Tested) 0:11:52

Llama-3.1 (Fully Tested) : Are the 405B, 70B & 8B Models Really Good? (Can it beat Claude & ...

Llama 3 405B 0:00:38

Llama 3 405B Releasing Today ⚡️

Metas LLAMA 405B 0:14:48

Metas LLAMA 405B Just STUNNED OpenAI! (Open Source GPT-4o)

Llama 3.1 | 0:14:41

Llama 3.1 | Meta is leading Open Source AI

LLAMA 3.1 70b 0:05:15

LLAMA 3.1 70b GPU Requirements (FP32, FP16, INT8 and INT4)

Комментарии
Автор

Please keep in mind that the context window also increases vram needs. 128k? We‘ll need something like Apple M8 Extreme Chip with X terrabyte(s) of unified memory. The cool thing, it will cost something around 10k-15k instead of 200k.

MeinDeutschkurs
Автор

I have 72GB VRAM - Can't wait to run the 405B parameter model at 0.01bpw.

But I am going to screw around with this on my 512GB RAM Epyc box. Expecting a couple seconds per token, should be wicked awesome.

Those_Weirdos
Автор

3:15 It Looks like we crossed the point when it was possible to run AI locally. Now, you need a tiny supercomputer to operate with cutting edge models. =((

Ukuraina-cssu
Автор

Look forward to your quantisation results😊

MrOktony
Автор

Awesome! Great video, learned a lot cheers 👍

InstaKane
Автор

can i use amazon or ibm servers to run the 70b or 405b model?

क्लोज़अपवैज्ञानिक
Автор

10:04 "You are on a Quuee" 🤭

martianreject
Автор

There was a sticker that came with your mic cluing you into the fact that it's a side-address 😆don't be a Yeti

threepe
Автор

I cannot get it install. I don't know what I'm doing wrong. I've gotten basically to every point. Except for the very last one when you type in y to confirm that you're okay with the file size

ThatGuyJoss
Автор

3:31 wait 3 years and it would be possible 😃

gileneusz
Автор

I tried to run the 70b model with 16GB and it just crippled the machine and ran up 2.5 GB of swap.

mendodsoregonbackroads
Автор

If anyone has about a quarter million they could loan me, I'll happily pay it back once I make all the Internet monies. Soon, I'll guess.

crs_net
Автор

NVIDIA A100 GPU is 30K USD, and you need many! Each GPU takes 450W, it's nonsense in electrical bill and initial price.

juliusvalentinas
Автор

Can I run it on my CPU? I have 44 cores and 512GB RAM.

thecount
Автор

I think you can try it on Hugging Face

dahee
Автор

8:42, not possible, the smallest quant is Q2_K 149.0 GB, possible to run on Mac Studio 192GB, but gives only 2 t/s, better to use just hugging

gileneusz
Автор

U need 250g ram to run the 4bit model.

ps