Fine Tune Phi-2 Model on Your Dataset

Welcome to my latest video where I dive into the fascinating world of AI and machine learning! Today, I'm excited to share with you how I fine-tuned the Phi-2 Model on a custom dataset. 🚀

About Phi-2:
Phi-2 is a ground-breaking Transformer model with 2.7 billion parameters. It's a successor to the renowned Phi-1.5 model and includes enhanced training with various NLP synthetic texts and carefully filtered websites. This ensures safety and educational value. Phi-2 has demonstrated near state-of-the-art performance in benchmarks for common sense, language understanding, and logical reasoning among models with less than 13 billion parameters.

Dataset Summary:
The dataset I used is a rich compilation of questions and answers from two leading online counselling and therapy platforms. Covering a wide spectrum of mental health topics, the responses are crafted by qualified psychologists. By fine-tuning Phi-2 with this dataset, my goal is to enhance the model's capability in providing mental health advice.

What You'll Learn:
✅ Insights into Phi-2 and its capabilities.
✅ Step-by-step guide on fine-tuning with a custom dataset.
✅ The impact of specialized datasets on model performance.

🔔 So, are you ready to explore the cutting-edge world of Gen AI with me? Watch the video, and don't forget to hit the like button if you find it informative. Your support means a lot!

💬 If you have any thoughts or questions, feel free to drop a comment below. I love hearing from you!

👉 Remember to subscribe and click the notification bell to stay updated with my latest content on AI and machine learning.

Thank you for watching, and let's embark on this AI journey together!

Join this channel to get access to perks:

#llm #generativeai #ai

[ravishmahajan]

Only one question, how do you learn all these? I mean what is your approach - you go through the documentation, code and then learn? How you have byhearted all the code parameters ?

[patrickblankcassol]

Can i finetune with other language dataset as portuguese?

[medec]

Awesome ! It would be a great help and more efficiant if you would not type but copy the code. And please always provide links to a colab notebooks. Are they on GitHub?

[the-ghost-in-the-machine]

such a great job and i appreciate your work and time. By the way, why you dont have a discord community channel, it would be great i guess..

[kevinyuan]

Such a great job bro！In the next video please add some steps to show us how to push the merged model to huggingface and then inference from hugging hub!🙏

[kevon]

Would love deeper dive into nf4. Enjoyed your intuitive analogy.

[DARK-fsrz]

please upload the inference part of this model as soon as possible .Thankyou

[jacehua]

ValueError: PhiForCausalLM does not support gradient checkpointing. currently getting this error I see that there are some other people mentioning this error to on the HF discussions.

[ajithshenoy]

Can you please provide insights on how to arrive at good hyper params for fine tuning ? Learning rate, number of epochs etc. Thanks

[tapanpati]

As you said you create another video and merge finetuned model and create gradio application, will you make it or not?

[hassubalti]

Thanks sir you for all your effort to teach us the fine tune our custom dataset...please mention that process which convet text dataset into question answer

[NawafNawaf-ppek]

Thank you!
I am trying to evaluate the model after fine-tuning, can you discuss how we can evaluate the model such as F1, EM and other metrics?

[dimioo]

The funny thing, that internet is full of those videos, where the code was copy-pasted from the HF over and over again.
But when I ask simple question from authors, like whether the specific model can be fine-tuned on my unlabelled data - silence 😂

And I notice a lot of issues with hyper-parameters, people are using for their different datasets, and it's causing huge loss and model hallucinations in the end

[mathnfact]

Do you have a GitHub repo or colab file for it?

[nurusterling]

Please make videos on to fine tune vision models

[tjskwgo]

How to find best fitting dataset on hugging face ? please respond

[Z]

Please make video on slm based ai assistant like Alexa. I think this is real work application for llms

[alroygama]

How can i take my multiclass classifications task and frame it to work with the setup u mentioned in the video?

Is it possible?

[IIT_YTT]

I have data set of indian constitution in CSV( discription, Articles) i want to fine tune phi 2 on this data.. Please Guide me

[tyxsutq]

but instrction format of phi is different why your using lama format?