Building an entity extraction model using BERT

In this video, I will show you how to build an entity extraction model using #BERT model. I will be using huggingface's transformers library and #PyTorch. You can use it to create any kind of #EntityExtraction model.

Please subscribe and like the video to help me keep motivated to make awesome videos like this one. :)

Follow me on:

[rayo]

I like your book it is straight to the point with handy code and I also appreciate the videos you make. I am a computer linguistics student and I have a neural network lecture where I have a project but we don't do any programming in the lecture itself so we all have to teach ourselves. this is greatly helping me understand the structure of a custom model. thank you and please continue spread knowledge. hare krishna everyone

[shaheerzaman]

Fantastic stuff as usual Abhishek. I bought your recently. 150 pages already done:). I think you should write a book on doing deep learning with Pytorch as well :)

[quandai]

Just wanted to say bought your book and love all your content!

[SP-ylbd]

Amazing tutorial. You're an amazing teacher. Just curious how this implementation is better/different from the NER pipeline from huggingface? Interms of accuracy, is it better to use this (the one you've explained) or is it better to go with NER pipeline? What are the advantage/disadvantages? Thanks

[Prashantsingh-xcho]

Thanks, i recently started working on bert and examples you have in your videos are great. 🙌
I would want to see how to handle a text file where there are no pos or columns of data 😵

[MangaGuy]

Great tutorial! I'm wondering how would you calculate training and test accuracy for this model?

[kumarsundaram]

Nice video, gotta learn alot! Thank you :)
Could you please also make a video on implementing Encode-Decoder model of Transformers package using any architecture like BART model for application like Neural Machine Translation?

[lukasnielsen]

Hi Abhishek, great tutorial as allways. One problem though; I was thinking that it might cause some issues that you extend the target_tag with 0's for the padding, since 0 is likely assigned as the encoded value for an actual tag. Am I mistaken in this?

[syedhasany]

Thanks for the brilliant tutorial! My guess is that we'll be seeing this again as a chapter in the "Approaching (Almost) Any NLP Problem" XD

[mehmetcalkus]

First of all, thank you for this useful sharing. I want to ask something.
Is the size of the fine tuned model obtained here the same as before it was fine tuned?

My question may be a little complicated, I just need your valuable ideas on how he could handle 2 different downstream tasks in a single model.

Actually, I also need a model that performs these 2 tasks, but is it more logical to implement them separately or in a single model? What is the size of the fine tuned model? Does it make sense to do it in a single model so that the application that the model will run on does not have memory problems(or or use less memory ), or should it be implemented separately to get more accuracy?

[fatvvsfatvvs]

Hi Abhishek,
Thank you! It's a great tutorial!

[lfmtube]

Thanks! very very clear. Bright teacher!

[Jeetkarsh]

Do you have any thoughts about training a NER model without a POS tags.

[shiyuyuan]

thank you for instructing, I have a question around 17:44, why add [0] (target_pos = [0] + ids + [0]) to the target_pos and target_tag? is that because the original value of target_pos/tag has been converted into numbers beside 0? but why add [101] and [102] to ids even we set special_token into false, because I remember the tokenize of [CLS] is [101] and [SEP] is [102]

[arijitdas]

Hi Abhishek, thanks for the great tutorial. However, badly need to understand how to deal with the subtokens? WordPiece is making the token level labels mismatched. Hence, hard to calculate any metrics such as Accuracy or F1. Please provide a solution. Thanks!

[anishsrivathsav]

A great Tutorial on Bert with text processing. I have one small question at 1:08:10. How to get back the tokens from the prediction of bio tags . Is there any reference to it as i didn't found any solution on web to get back the token from the predicted tag.

[kmnm]

Hi Abhishek,

Very incisive video on NER with BERT . Excellent. Just a thought : I am comfortable with TensorFlow and not with Torch. Is there a TF version of the code which will help me immensely? Krish

[ardavaneghtedari]

I typed the code as you presented in this video but I receive an error: error 32 broken pipe and it happens when the tqdm loop starts. Any change needed in the code?

[prakashkafle]

How BERT can be use for long text ? Due to 512 token limitation and tokenization based on word peice it's difficult to address long text .

[AAAA]

amazing video! Where can I learn to assign a word to entity, you mentioned but did not explain at the end?