Llama 3 on Your Local Computer | Free GPT-4 Alternative

In this video I will show you the key features of the Llama 3 model and how you can run the Llama 3 model on your own computer. First, I will cover Meta's blog post in detail to explain what's new about the Llama 3 model, and then I will show you how to run the Llama 3 model on your own computer in a very simple way. To do this, we will use the vLLM and Gradio libraries, which will allow us to interact with the Llama 3 model in a visually appealing and lightning-fast way. I hope you'll enjoy how easy it actually is! Let me know in the comments what you think about the new Llama 3 model. :-)

My Medium Article for This Video:

Github Repository

Request Access To Llama 3

Llama 3 Blog Post

My Workstation

Web Highlights - Chrome Extension

00:00:00 Intro
00:00:20 Key Changes
00:01:55 Meta Blog Post
00:04:13 Model Architecture
00:06:43 Training Data
00:08:00 Scaling Up Pre-Training
00:10:13 Instruction Fine-Tuning
00:12:02 400B+ Model Is Coming
00:14:03 Llama 3 On Your Local Computer

Stay in Touch

Medium

LinkedIn

YouTube
Of course, feel free to subscribe to my channel! :-)

Of course, financial support is completely voluntary, but I was asked for it:

[nartrab]

Thanks for sharing. It is amazing that not only you create quality videos, but you also reply to so many technical problems. You are a great guy.

[EwenMackenzie]

thanks for sharing! this was super helpful :D

[codingwithsarah]

hello, just wondering . Can you help in doing this on Google Collab ?

[metanulski]

Very Nice. Since not everyone has 40 GB Vram, can you be more specific on how to do this with the llama3 8B model. ( because you say we maybe need to change the datatype ist we use a different model, and I have no clue how I should know the correct data type 😁 )

[stresstherapist]

is this on windows or linux. can't seem to install the vllm library on windows

[jennilthiyam]

Hi. Thank you for your video. I wanna know one thing. I have a multiple CSV files which I want the llama to know about it. I have went through other videos, there is a guy that does the same task like I want but after incorporating the files, llama cannot respond other general question correctly but focus only on the information of CSV file. Their method first split the text into chunck and use embedding to embedded them using other embedding methods. Can you please provide any solution to it using only llama and nothing else. What I want is for the llama to know about my files in the top of its already existing knowledge.

[stefanocianciolo]

If i wanted to only get the text results and not launch the UI, what should I remove? Thanks!

[jeffbruno]

When i run pip install command I get the error "Could not find a version that satisfies the requirement flash-attn==2.5.7"

[allo.allo.]

How to run llama 3 70b on 4 x rtx in linux?

[Stealthy_Sloth]

I need the model to upload files for anaclasis like Chat-GPT's interface.

[mike]

Can I run the 70B with an rtx 3090, has 24gb of vram, and how would I do it?

[mohsenghafari]

thanks . is great job.
can i use for gpu 3080?

[tubebility]

I too have an RTX 6000, but only in my dreams. 🤑

[shotelco]

Asking a LLM questions is fun and everything, but most will want a LLM to act as an "agent base". Utilizing a multi-expert foundation. Meaning the LLM is tasked with a coding problem or a finance problem, or to re-write a story. The LLM base is where the agent front-end goes to. How about you front-end something like pythagora;DOT:ai using Llama3 as a LOCAL backend over API? And (and I know I am asking a lot here), provide a training methodology which ingest something a like companies FAQ's, help-desk/knowledge base/ etc?

Otherwise playing with any AI is more amusement and entertainment than an actual system for productivity.

[ss]

Thanks! But lol, joe average don't have a 10, 000+ euro GPU :D

[avi]

let's be real, it's not GPT-4. I don't know why people insist on trying to make this false equivalency. No open source model has still come even close to GPT-4. They can release all the benchmarks they want and blah blah blah, using the two models you immediately see that llama3 is still quite a bit weaker than GPT-4. We'll see when the 300B version comes out. I'm not holding my breath though. If 300B still falls short then it wil be at least another year and a half maybe two before llama 4 comes out that should finally surpass it but by that time GPT-5 will be out and llama will again be behind of course.

[strategy]

can i getvyour email for business inquiries