LLM Crash Course Part 1 - Finetune Any LLM for your Custom Usecase End to End in under[1 hour]!!

Unlock the secrets of LLM finetuning with this comprehensive crash course! Learn the two-stage approach, from Supervised FineTuning (SFT) to Preference Alignment, and discover how to tailor your model to perfection. 🚀💡

Embark on a journey of LLM finetuning mastery in this crash course, where we dive into the intricacies of optimizing your language model for any use case. Here's what you'll explore:

Two-Stage Finetuning Approach: Delve into the methodology of the two-stage finetuning process, starting with Supervised FineTuning (SFT) to align the model with initial instructions. 🎯📈

SFT Phase: Learn how to fine-tune the model using Supervised FineTuning, where instruction data guides the model's behavior to adhere to specific user directives. 📝🤖

Preference Alignment: Discover the next phase of finetuning, Preference Alignment, where various methods like DPO, IPO, and KTO_pair are employed to customize the model's responses according to user preferences. 🔍🎯

Optimizing Model Performance: Gain insights into optimizing your model's performance through iterative refinement and experimentation, ensuring it generates responses tailored to your exact requirements. 🛠️🔬

Join us as we demystify the art of LLM finetuning and empower you to craft language models that truly resonate with your needs and preferences! 💬✨🔧

Important Links:

For further discussions please join the following telegram group

You can also connect with me in the following socials

[shaigrustamov]

You made my weekend. I found your videos today and they were great. Thank you so much! 👍🏼

I have a question. I have fine-tuned Llama 3.1 8B model for data extraction with the OCR texts of the invoices. For fine tuning I used “llama recipes” and dataset I prepared like samsum dataset. But the results does not look so good. Since in accounting have different approach, the invoices are very different. For data extraction what would you recommend? Which model would be better than Llama for this?

[malathimurugesan]

Thanks for the video. Easy to understand. Why self.model is passed as parameter and the return value is the same, self.model.? Self.model is not initialized as a constructor.

[shrawaniambhore]

Please make video on a project to train the data using llm and get the percentage match between resume and job description.

[bennguyen]

I notice some of the models at HggingFace are huge (over 25GB!)

Do these models require training? If not, but you wanted to train it with custom data, does the size of the model grow, or does it just change and stay the same size?

And people say its expensive to train your own data, is that because of the power consumption and/or the cost of using cloud GPUs (aws, etc)?

[naveenpandey]

I have json data do i newd to use different method to fine tune model for my data?
Or i can use this code to fine tune on my data

[criticalnodecapital]

I wanted to know, if I have the whole Legal corpus from the last 50 years of India’s legal cases. I want it then fine tune my llama 70B into this days set.

I want to do MLL in understanding and then also responding in legal way.

- use IBm debater data set
- use HF other data set that help in reasoning

Then Perform QLORA for the training and get parameters perfect.

Then I want to do supervised learning by having the data set cleaned up when it comes to finance law etc.

Now that the training is done, and fine tuning has finished. We are ready to use the model..

I wan to be able to use the INFERNENCE API from Facebook.

I think about new Rags systems go implement and have the perfect chat completion bot.

I will soon it will be ready to share.

[tintintintin]

Bro, did I get this correctly that while using DPOTrainer we can use different loss_type values? i.e. kto_pair, sigmoid, hinge, ipo

Or am I supposed to use KTO/IPO specific trainer, in case I want to use loss type kto_pair/ipo?

[aibasics]

will the mistral quantized model work for fine-tuning..like GGUF model.

[naveenpandey]

Bro I am using mistral model for generating summary as of now I am getting response in 50 seconds and i m using t4 gpu is there any methods or way to get response within 20 seconds .
Please share resources.

[hemachandhers]

The `load_in_4bit` and `load_in_8bit` arguments are deprecated and will be removed in the future versions. Please, pass a `BitsAndBytesConfig` object in `quantization_config` argument instead. i have this error

[arnabmaity]

Sir I want to Fine Tune StarCoder v2. How to do it??