Python with Stanford Alpaca and Vicuna 13B AI models - A llama-cpp-python Tutorial!

In this tutorial chris shows you how to run the Vicuna 13B and alpaca AI models locally using Python.

he explains the differences between vicuna and alpaca and shows you how to download the vicuna model and then how to install llama-cpp-python on your machine and create a basic python app that allows you to query both the vicuna and alpaca models, comparing the differences

if you want to build python apps against AI LLM's this is the video for you.

[IronZk]

The llama_cpp is a very cool way to use thoses weights!

[SantiagoRestrepo-zf]

I have this problem: 'Llama' object has no attribute 'ctx' could you please help me

[SCNewbCaster]

This is exactly what I have been looking for.

[SMCGPRA]

Now a days, I am blindly subscribe, like and save whichever video for AI, chatgpt, vicuna, koala, langchain, llama etc...

[hyperconqueror]

Hey, I am doing it on windows
And when I am loading the model, assertion error comes in the line

assertion self.model is not None

[marius]

I believe the low quality answer is the result of the low quantization of the model (4bit). Even the 7B un-quantized vicuna (without llamma-cpp) gives a much better answer:
USER: Who is Ada Lovelance?
ASSISTANT: Ada Lovelace was a British mathematician and writer, who is often considered to be the world's first computer programmer. She was the daughter of the famous poet Lord Byron and wrote a series of notes, now known as "Ada's Notes", in which she described a proposed mechanical general-purpose computer designed by Charles Babbage. Lovelace's notes included an algorithm to calculate Bernoulli numbers, which is considered to be the first example of computer software. She died in 1852 at the age of 36.

Notice that the model autocorrects the spelling of the name.

[sallymuhammad]

whenever I run the pip install llama-cpp-python, I get this ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects. Does anyone know how to solve it.

[erivgout]

thanks you so much for this have been looking for a tutorial

[soonheng]

Typo at 6:25, "Ada Lovelance", that is why Vicuna not sure who she is I guess.

[zacharypump]

Why do I need llamacpp? Can’t I use regular huggingface / langchain python bindings to run these models??

[cataclismo_capoeira]

Awesome video brother. I like the way you teach

[HighOnQuack.]

How would I make it act more like a conversation, AKA make it like chat gpt instead of it completing the text i put in, it generates a response to the prompt?

[jacky]

AttributeError: 'Llama' object has no attribute 'ctx'
whats that mean? is it version issue?

[picklenickil]

I think the question you are asking are more knowledge and expertise based questions.

Imma try giving it long text to summerise and see.

[barbarbarbar]

Hi, thank you tor this tutorial.
Does somebody find how to run it on GPU ? I have Cuda correctly installed but this model (vicuna) is always running on my CPU I don't know how to deal with that (and it answers very slowly ... between 5 and 45 seconds)
Thanks

[mallorywestwood]

Thanks! Do you have a follow up where you ran this on webserver and fine-tuned the model?

[KanwalKhalid-xs]

i am getting error while loading model " assert self.ctx is not None". It would be pleasure if you help me to get rid of this error.

[AlienAnthony]

Were you able to get GPU loading? I'm having trouble getting it to load in anything but CPU

[vinu_s]

Do you have a video that explains how to train vicuna model with custom data?

[someoneisasking]

Hi im getting an error for llama python installation. I am getting a Building wheel for llama-cpp-python (pyproject.toml) ... error