How to build custom Datasets for Text in Pytorch

In this video we go through a bit more in depth into custom datasets and implement more advanced functions for dealing with text. Specifically we're looking at a image captioning dataset (Flickr8k data set) with an image and a corresponding caption text that describes what's going on in the image. I think the general principles from this video can be utilized to any project you're working with when dealing with text data be it either translation, question answering, sentiment analysis etc. I also recommend taking a look at my Torchtext which can also be quite helpful and simplify the data loading process.

❤️ Support the channel ❤️

Paid Courses I recommend for learning (affiliate links, no extra cost for you):

✨ Free Resources that are great:

💻 My Deep Learning Setup and Recording Setup:

GitHub Repository:

✅ One-Time Donations:

▶️ You Can Connect with me on:

OUTLINE:
0:00 - Introduction
2:05 - Overview of what we're going to do
4:05 - Imports
5:20 - Setup of Pytorch Dataset for loading Flickr
11:50 - Setup of Vocabulary and Numericalization
22:19 - Creating Collate for Padding of Batch
25:20 - Function for getting data loader
29:15 - Running code & fixing couple of errors
33:09 - Ending

[tomkohler]

This will save my live. I try loading data since one week and only fail.

[sachavanweeren]

Thanks for the tutorial. It might be worthwhile to show intermediate results of what different parts do earlier in the video to show exactly what certain code snippets do

[foobar]

Very useful source code. Shouldn't remember it by heart, but worth to understand.
Thank you!

[haideralishuvo]

Awesome tutorial, best channel on pytorch :D

[vijayendrasdm]

Thanks for the video. This is helpful.
Waiting for the next.

[aboalifan]

Thanks Aladdin, best Pytorch tutorials on the web

[sayedathar]

Thanks Alot for your videos it is helping me alot to learn pytorch, I am trying out to build an Image Captitioning model on a Custom Dataset, Your Videos on Image Captitioning will be useful alot :), Thanks alot again

[deepshankarjha]

this is the best pytorch tutorial on the internet. even better then the doc provided by the website

[takagisa]

It's a really nice tutorial, thanks a lot!

[orjihvy]

In future videos, may be you can also add an explanation as to why you architected the objects in this way.

[mitable]

we can also use the torchtext Field class for the EOS and SOS and in the same class we have build vocab too

[sayedathar]

This was really helpful thanks alot bro your videos are saviour Love You :)

[curatorsshelf]

Thanks a lot for the video. Update for the spacy configuration:
spacy_eng = spacy.load("en_core_web_sm") - is the correct way to do now :)

[thecros]

Really enjoyed the learning journey with u❤️❤️

[gaurikmukherjee]

Amazing video! Had one doubt. Does spacy remove punctuations and white spaces, because it is not doing that when I am trying?

[TheFotbollen]

Great video! Do you have an idea of how to translate from english to python code (with custom dataset) using transformer?

[ahmetsuna]

Great video Aladdin. Thanks. I have one question: at the last of the video, sequence lengths seems different. Why they do not equal to [26, 32], isn't that a mistake?

[simoneparvizi]

The video is great, really. Just 1 thing that (personally) would make everything literally perfect: could you explain literally everything? Like when you mention transform at 5:50 and you said that you put it as None, explain why etc. As well as for the rest. Basically what you did at 6:40 for the "csv" function explanation. Again, this is only my personal opinion and it would personally help me so much


Keep up the great work!

[hrithicksen]

Bro, please implement more papers. Make a video on How to use YOLO in torch... Please dude

[adesiph.d.journal]

Amazing Tutorial. Thanks for it! I am missing the need .unsqueeze(0) for each item in the batch while assigning it to the imgs. Any input on that would be much appreciated. Thanks!