Embeddings vs Fine Tuning - Part 2, Supervised Fine-tuning

*Supervised Fine-tuning Notebook*
- Run fine-tuning using a Q&A dataset.

*Fine-tuning Repository Access*
1. Supervised Fine-tuning Notebook
2. Q&A Dataset Preparation Scripts
3. Scripts to create and use Embeddings
4. Forum Support

*Other Resources*
Presentation Slides:

Llama 2 Inference Notebook (to compare Chat and Base models):

0:00 Part 2: Supervised Fine-tuning
0:30 How to make fine-tuning work?
1:10 How NOT to do fine-tuning
2:20 Video Overview
2:50 Chat vs Base models
7:50 Supervised versus Unsupervised fine-tuning
19:40 Converting a dataset into Q&A
30:50 Supervised fine-tuning in google colab.
46:45 Pro tips
48:32 Scripts and GitHub Repo Access

[EvntHorizon]

Appreciate the illustration of the difference between base and chat models.

[simonstrandgaard]

Excellent and well presented. Different than other fine tuning tutorials. I appreciate that it's an unfamiliar topic "Touch Ruby" that it has no knowledge about, that's interesting seeing how it progresses. Great job.

[mark-pwxf]

Enjoyed your series of tutorials. Thanks!

[abadidibadou]

Keep up the good tutorials. you are doing a really good job

[rbdon]

I just wanted to let you know that these videos are really fantastic compared to many of the other ones I've seen. I really appreciate it!

[alienorreads]

Going to a interview right now. This has been really helpful for remembering and learning the logic. Thank you so much❤

[carydunn]

awesome job explaining everything in extra detail

[ghrasko]

From other sources I hear that one uses finetuning not for knowledge teaching, but for improving process following or output format (if prompting is not giving satisfactory results). They suggest using RAG for knowledge improvement. What are your thoughts about this?

[TheRealRoySadaka]

Great content, appreciate 👏🏼
There are many QLoRa tutorials out there, including some “official” ones, but I haven’t seen mask handling in them as you describe, they use SFTTrainer, yet, what you present with the custom trainer makes perfect sense, does this means the rest of the tutorials miss the mask handling? Or is it baked into the SFTTrainer already?

[Mohamed-sqod]

great content really helpful, quick question if the model doesn't have access to the whole rule book documents, how could he reason to answer questions other than the ones given in the train data ?

[aasembakhshi]

Thanks for this video as I keep coming back to it. Here is a scenario.

If we have a working RAG with an open source model, (let's say, Gemma-7B) and we have a fixed corpus (let's say a pdf book), would it be a good choice if I replace the open source model with a fine-tuned version tuned on my corpus?

I guess it should work better than the simple RAG. But now I have second question. For fine tuning such a model, should I have a data set with three columns: Chunk-of-Text, Question, Answer? Or should I have two columns (Prompt, Response), with the prompt including the chunk-of-text as the context as well?

[AIToluna]

Well done!
Purchased the notebook, hopefully it will help support the channel.
Quick question: in "prepare_dataset", response_lengths and input_lengths are lists of ints, which then gives an error in TextDataset __getitem__,
This is because the "idx" parameter in __getitem__ is a list (size of batch_size), and not an int, so doing
self.input_lengths[idx]
gives """TypeError: list indices must be integers or slices, not list"""
Even if batch_size=1, the "idx" parameter is a list with 1 element, and still getting the same error, am i doing something wrong?
Or should the input_lengths turned into tensors to support list idx? thank you in advance

[okj]

Would it be better to do orca style prompts for q/a dataset?

[yusufkemaldemir]

What do you think of using embeddings+supervised fine tuning? Thanks

[darioai]

ins't stuff like data generation against openAI ToS?

[aneerpa]

Really informative, your channel should get wider traction.