Fine Tune Transformers Model like BERT on Custom Dataset.

Learn How to Fine Tune BERT on Custom Dataset.
In this video, I have explained how Finetune transformers models like BERT on the custom dataset. How to use hugging face Trainer API, saving and loading fine-tuned model, evaluate the model on validation dataset, and making model prediction on a single example.

NLP Beginner to Advanced Playlist:

I am a Freelance Data Scientist working on Natural Language Processing (NLP) and building end-to-end NLP applications.

I have over 7 years of experience in the industry, including as a Lead Data Scientist at Oracle, where I worked on NLP and MLOps.

I Share Practical hands-on tutorials on NLP and Bite-sized information and knowledge related to Artificial Intelligence.

#machinelearning #artificialintelligence #datascience #nlp #bert #transformers

[FutureSmartAI]

📌 Hey everyone! Enjoying these NLP tutorials? Check out my other project, AI Demos, for quick 1-2 min AI tool demos! 🤖🚀

We aim to educate and inform you about AI's incredible possibilities. Don't miss our AI Demos YouTube channel and website for amazing demos!
Subscribe to AI Demos and explore the future of AI with us!

[athariqraffi]

Thanks for the video, I can understand easily from your explanation.

[infrared.]

I searched lot read lot to solve one simple compony assessment problem but not able to solve...it
...as wont find any fine tunning video.
You are gem

[mansibisht]

Great video!!! You just solved a proposed RFP at my work. Thanks Pradeep!!!

[ashishmalhotra]

Hey Pradip. Your videos are very informative. Just a suggestion, instead of putting chapter numbers can you put a small description so that one can jump straight to the desired timeline

[adekunledavidgbenro]

Thanks for this video. Really helpful.
Can you do a similar video for pretrained NMT model for let’s say Danish language?

[jacobpyrett]

GREAT video! solved exactly what I was looking for.. thanks so much!

[bassemgouty]

very nice video and well explained, well done !

[Tiger-Tippu]

Hi Pradip, whats the purpose of creating Pytorch custom dataset when we already have our own dataset

[Iiochilios]

Is it natural way to create custom dataset?! Can't believe you have to write custom class for this simple task.

[OnLyhereAlone]

New subscriber here.
Thanks for this clear explanation. I have watched a couple other videos of your and still watching but i have this question that you did not get to in this example because you had only 1 epoch. If i trained say for 10 epochs while tracking metrics (e.g., validation loss, accuracy or F1 score), if my best model was arrived at at the 6th epoch, how do i specify saving that 6th epoch?
Thank you.

[_bimandas]

Hey Pradip, for News Summarisation project can I fine-tune BERT with CNN/Daily dataset ?
Will this perform better than the basic BERT model ?

[DivyaPrakashMishra]

Followed the same approach but getting this error for trainer.train() method

Expected input batch_size (1360) to match target batch_size (16).

[tehzeebsheikh]

Hi can we use the same code for distilbert or roberta as well?

[saadkhattak]

HI pardip, I was following your code and got this error

Target size (torch.Size([8])) must be the same as input size (torch.Size([8, 2]))


can you help me fix it? I was simply running your notebook in google colabb

[AK-wjbx]

Hi @Pradip Nichite , Thanks for the great explanation :)
I have a question: I have a machine generated data which is not natural language(Although the sequence of words in the data is important).
I do not have any labels in the data, would it be wise to fine tune BERT and generate word embeddings using BERT?

The idea is to check if BERT would generate more meaningful embeddings when opposed to word2vec skip gram.
Thanks in Advance :)

[victorwang]

Great explanation and the notebook works! I followed the notebook and fine-tuned a BERT model. I found two ways to use the model: tokenizer =
model = BertForSequenceClassification.from_pretrained('custombert', num_labels=2) ; tokenizer =
model = Either way, I can't load the tokenizer. Is this because I didn't update the vocabulary? And what's the difference between and Thanks a lot!

[AlexXu-csbt]

Hi Pradip, thank you for this tutorial. Is it possible to fine tune the BERT model to predict a multiclass output? For example, emotions rather than a binary classification like this example.

[eqjjcsx]

Hi Pradip, thank you for this tutorial.
I just want to ask you that do you have any tutorial for fine tuning BERT (or BERTology methods) for GENERATIVE question answering task? Hope you can see my comment. Thanks in advance!

[harrylu]

Hi Pradip, this is a great video. Thanks for your efforts to create this for us. Could you please give me some advice to tackle the data privacy issues when using these pre-trained model from hugging face? I understood that when we import these pre-trained model and do training, we might be sending the private data that we are training through API? Based on your experience, if we want to secure the data from public but still enjoy the benefits of these pre-trained model, what would you recommend? I know hugging face is promoting their private hub demo. What do you think about that?