311 - Fine tuning GPT2 using custom documents​

311 - Fine tuning GPT2 using custom documents​

Code generated in the video can be downloaded from here:

All other code:

This tutorial explains the simple process of fine-tuning GPT2 using your own documents. It also demonstrates the advantages of structuring your training data as Q & A rather than long text.

[goncaavci]

You are perfect to teach And I like your English. Thank you

[ChiaHuaLee]

Could you also share the dataset please?

[dcdcdc]

Hello, one doubt, the question "What is the Babel fish?" in defined explicity in the file custom_q_and_a ? the users only could ask questions that was defined in the file custom_q_and_a? Thanks for your content are very interesting and clear

[muslim_bro]

Amazing how easy gpt2 can be fine tuned. My question is, do the final result can handle different questions in same context?
Who is the babel fish?
Tell me about babel fish?
Do you know babel fish?
Etc..

[kazeemkz]

Fantastic, just what i was looking for !

the question is, was that same question in the training set with a similar response?

[prep]

Hii Sreeni! I loved your video, which gave me a lot of insights to learn the LLM model and fine-tune it over my own dataset. When I fine-tuned my own q_and_a dataset, I didn't get a good result. Can I have access to the article where you trained your model?

[m-aki]

Very good and useful tutorial. I tried this as in the tutorial, and trained the model to create an inquiry service chatbot on my campus. but the answer given is still not relevant, is there a problem in the dataset? Maybe you can tell me what the minimum dataset is needed or what optimizations affect the ability of this model?

[gustiayuwahyuwhurapsari]

can you provide the structure of q n a dataset? thank you

[ritubansal]

Hey, the tutorial is quite interesting but I'm having a challenge to finetune the model on 50 epochs and the dataset is around 22MB how much it will take the time to train using GPT2 model.

[raihanpahlevi]

why you didnt show how the train data looks like after preprocessing?

[anammanzoor]

Hi Sreeni can you please make one video on ViT Swin transformers implementation?

[superfreiheit]

very confusing code; Where is the train dataset ? How is the format. very importing thing missing

[souravpal]

I have one question...how do I add memory to the model, ...in other words, how do I allow the model to remember older conversations?

[winayazarkasih]

Is there any example of dataset Sir, Your video is very helpful but I have struggle to make or find the dataset

[World-Of-Mr-Motivater]

sreeni can you give us the link for the q and a document ?
it's not there in your github

[dhairyakataria]

First of all your tutorial is very useful, there is very little materials on net related to this.
I had some questions,
like can we use this same approach for different models like flan-t5 or bloom, by just changing the model name,
Is there any other way to train model on only text data and not question/answer?
In this method is the model adding new parameters to the pre-trained model or is it doing something else.

Thanks in advance

[devtest]

Thank you for this!! I ask you a question if I wanted to do this but with more than 10 thousand documents, would it be better to use a vector database and do similarity queries?

[Shehab_Zexas]

your tutorials have been a greet help for me during my college!
i have a small question
if i want to train it for text summarization do i follow the same steps ?
my data set is text and summary .. should i join them in the same file as [text] then [summary] like we did in the questions ?

[melaneemelanee]

Where we can see your dataset? if it is possible, please publish it, Tnx