How To Run Llama 3.1: 8B, 70B, 405B Models Locally (Guide)

Показать описание

Meta has finally released Llama 3.1 models which feature: 8B, 70B and a massive 405B model which is able to compete with the state-of-the-art closed source models. In this video I will show you how to run and deploy Llama 3.1 models locally on your laptop, along with their requirements.

School of Machine Learning

Рекомендации по теме

Комментарии

good, I will now try to run the 405B model on my 50, 000$ pc.

ecchiRhino

You can run the 405b model on a server with at least 256gb of ram and a vega 56 or 64 graphics card.
That specific card can access ram as if it were vram and on some platforms, bypass the CPU.
You can also buy optane pm for cheap. Either adding 4x4 optane modules via pcie then raid 0 and add the raid to swap, or as ddr4 dims to just add as a massive ram pool.
I'll be experimenting with Vega 20 next week

flpiutc

um 405b is like 300 gigs good luck with THAT lol

gu

Thanks a lot for the detailed explanation in the video! I have a question regarding Ollama. Is it possible to use Ollama and the models available on it in a production environment? I would love to hear your thoughts or any experiences you might have with it. Thank you!

is there a single source of Information which could give details hardware requirements for each of Llama 3.5 Models (i.e. GPU, RAM, Memory, Cache Memory etc.)

deepaksingh

Have u guys tried 405b version localy ? What are ur pc specs ?

ouso

How To Run Llama 3.1: 8B, 70B, 405B Models Locally (Guide)

How to Run Llama 3.1 Locally on your computer? (Ollama, LM Studio)

How To Run Llama 3.1: 8B, 70B, 405B Models Locally (Guide)

Llama 3.1 is ACTUALLY really good! (and open source)

How To Run Your Llama 3 1 Models With Open WebUI Web Search Locally

Run Llama 3 on Windows | Build with Meta Llama

Zuck's new Llama is a beast

How to Run Llama 3 Locally on your Computer (Ollama, LM Studio)

Install and Run Llama 3.1 LLM Locally in Python and Windows Using Ollama

Running Llama 3.1 8B on MacBook Air 8GB using Ollama

Run Llama 3.1 locally using LangChain

EASILY Train Llama 3.1 and Upload to Ollama.com

How to Download Llama 3 Models (8 Easy Ways to access Llama-3)!!!!

Start Running LLaMA 3.1 405B In 3 Minutes With Ollama

Correctly Install and Use Llama 3.1 LLM in Python on a Local Computer - Complete Tutorial

How to use Llama 3.1 ?

How to Easily Install and Run Llama 3.1 on a Local Windows Computer -Meta LLM alternative to ChatGPT

Build Anything with Llama 3 Agents, Here’s How

'okay, but I want Llama 3 for my specific use case' - Here's how

Build Anything with Llama 3.1 Agents, Here’s How

This Llama 3 is powerful and uncensored, let’s run it

How to Install and Run LLAMA 3.1 8B model on your LAPTOP with OLLAMA

Install & Run Llama 3.1 in 2 min on Windows Locally

How to Install and test LLaMA 3 Locally [2024]

Llama 3.1 405b model is HERE | Hardware requirements