Fine-tune LLama2 w/ PEFT, LoRA, 4bit, TRL, SFT code #llama2

preview_player
Показать описание
Code script how to fine-tune LLama 2 model with parameter efficient fine-tuning, a low rank approximation of matrix and tensor structures, a 4-bit quantization of tensors, a transformer based Reinforcement Learning (RL) and HuggingFace's Supervised Fine-tuning trainer. LLama v2 model, finetuning.

Plus we code a synthetic dataset for our LLama 2 model to fine-tune on, w/ GPT-4 (or your preferred CLAUDE 2 or ....) as the central intelligence - to create task specific datasets for a given user query to fine-tune LLMs on.

All rights with Matt Shumer for his Jupyter NB on fine-tuning LLama 2 model:

See also Matt Shumer's Github repo for the GPT-LLM-Trainer:

#gpt
#finetuning
#llama2
Рекомендации по теме
Комментарии
Автор

Fantastic! Appreciate the knowledge you are sharing.

lifsys
Автор

Bro, I appreciate you so much for this fire content you been pumping out, after checking you out over the past week, you have gained a subscriber for sure. Great stuff, please keep this up!!

lifeofcode
Автор

Is it possible to train, in the same training go, a dataset made of prompt/response and full text files?

echofloripa
Автор

Awsome content! When is it adecuate to fine tune an llm instead of working or as a complement for the botpress knowledge base?

elrecreoadan
Автор

Do you have a discord community? I have been following you for awhile now and have so many questions. BTW this is amazing but I really want to talk more about instructor embeddings FAISS db and instruction fine tuning something really small like flan t5 small/base. I'm curious on if with peft lora ability to freeze and manipulate the weights of the base model would we be able to run a real form of intelligence on a cpu? I know the amount of data would be a lot but would we be able to see Fair results? Sorry in advance if this is wrong place for this question

dustingifford
Автор

Thank you for this Video!!
I'm new to fine-tuning and trying to understand more about it. Can someone explain if test and evaluation datasets are needed for instruction datasets? I'm not quite sure how test and evaluation datasets work with instruction data. Additionally, I'd love to know what's the best percentage split for instruction fine-tuning on a dataset of 5K rows. Would a 10-10-80 or a 20-20-60 split be more suitable? Any advice would be greatly appreciated!

moonly
Автор

So In reinforcement learning, the reward model was LLama 2 itself or chatgpt4?

akeshagarwal
Автор

how long did it take to run the collar notebook, using T4 GPU or TPU?

wryltxw
Автор

Thanks for sharing.... do you know if this one can be tuned to 8bit. the one you mentioned to 8 bit does not applies to this.

MLesp
Автор

Why we need to merge the model again in the last stage?

hunkims
Автор

Can I run the Colab NB on a free account?

echofloripa
Автор

Channel: "You know this..."
Myself: "nooo, I don't, go back... " 😅😅😅

echofloripa
Автор

Could we do this without openAI and off of something completely offline?

redgenAI
visit shbcf.ru