Installing Llama 2 on Windows Using oobabooga Web UI

Downloading the new Llama 2 large language model from meta and testing it with oobabooga text generation web ui chat on Windows.

Note regarding the token:
from: @user-tc5cc3or5m
with the recent oobabooga version, you have to set HF_TOKEN with your token only. if you set HF_USER and HF_PASS as shown in the video, things won't work anymore!

If you get the 401 Unauthorized error in the UI tab even after creating the environment variables, then try either of these 2 alternatives:
1. do a git clone to a new folder within the models folder. Like create this folder "oobabooga_windows\text-generation-webui\models\meta-llama_Llama-2-7b-chat-hf" and within that do the git command from hugging face:
git lfs install
That should result in the same files.

2. You could also manually download the files from the hugging face Files and versions tab and place them in that folder: "oobabooga_windows\text-generation-webui\models\meta-llama_Llama-2-7b-chat-hf"

[foreignconta]

The way the tts pronounces Uuuuga boooga with a British accent. 😂

[iammyself]

I agree with the others that this is the only walkthrough so far that makes sense for installation on Windows. Thank you!

[extension]

Best shortest tutorial ever on YouTube

[TerkelBrix]

I LOVE the energy! Feel like I'm witnessing something spectacular

[MrAlex]

You are the best. On Mac and Linux installation is quite easy and well documented. But on Windows this is the only tutorial that made sense.

[adrianfiedler]

Love your style. Sounds exciting like a boxing match.

[kamikase]

Hi, in the task manager, change the option Copy to Cuda, to se the actual usage of the cuda processors

[Galactiac]

Still can't get this to work. I keep getting errors when I try to install the model.

[thehkmalhotra]

Thanks a lot mate for this video. I literally loved it. Can you please lemme know how to build API for this?

[snehasissnehasis-cosn]

How to deploy anything -llm on docker for windows pc.... Please make a step by step Video for that project...With attached all the code....

[The_Real_James_Bragg]

I have a pretty beffy server setup. E5-2699 20 core 40 thread, 128gb ddr4 and Nvidia 3090 24gb vram. What would be the best os to run all this on?

[dannydro]

Love the accent, still trying to get used to the voice tone. It's like listening to a radio transmission of a Football game. I keep waiting for

[keshavmadhavan]

Your voice has an energy of an army parade! an AI army parade!

[Winter_]

The GPU required to run this should have very high VRAM, my 4 GBs of VRAM are struggling to run this. Great video though ! :D

[glnxcqw]

you are the best person, will you create a video for running LLAMA on AZURE virtual machine

[Pichetk]

I've GTX 1060, 4 GB . So how can I ran the Llama 2. Last time, it said I had not sufficient memory.

[CHURCHGPT]

What if we don't have a GPU? Been having a hard time figuring out how to setup and use these models on cloud instances. Sincerely - Noob

[yjj]

I set up the 2:20 video well, but the hugging face is not installed and the error continues. I don't know what's wrong

[MrAlex]

I still fail - lol. I did setup the env. variables as shown here one entry HF_USER one HF_PASS, I created and read access token on HugginFace and copied USername in HF_USER and token in HF_PASS. Even after saving it still gives the exact error 401 Unautorized. Any ideas?

[The_Real_James_Bragg]

Did not work for me. I still get the Unauthed URL