Python tips and tricks - 10: Loading images and masks in the right order for semantic segmentation

This video explains the process of loading images and masks in the right order (in python) for semantic segmentation .

Code generated in the video can be downloaded from here:

[ajay]

from this video, I can say that you cover more than needed. Awesome trick saved my time. Thanks a lot sir

[emransaleh]

I like how very technical are your videos in core computer vision tasks. Keep it up same quality.

[vangetinikhil]

Actually in windows OS this random loading of images takes place. Basically organizing images before working on them is of ample importance. Actually there are several ways of sorting images in the order we want. Your way is absolutely fine and easy. There are other ways. I use the module called 're' and also making use of "lambda" function operator and also use sorted() inbuilt function of python. Also I write a line of code to see the list of files and call it as "fil_list" as follows:-
fil_list = sorted (glob.glob('PATH'), key=lambda x:float (re.findall(("\d+)", x)[number])).

Here the "number" is integer value number which is upto user and its significance is that if you have your files in single folder then you should write number as 0. If files are after some nested folders then number is the exact number of nestings.
For instance our file is having path structure as: 'D:/folder1/folder 2/files", then number = 2.

[anandsingh]

@Srini- you sort really saved my ass ;)
I am usually working on windows and now I have GPU Ubuntu machine for my massive image analysis. I watched this video and forgot, but when my files were all random and especially- images and corresponding masks! Great thanks to you!
good work🤘👃

[jacobusstrydom]

Fantastic solution to a frustrating issue. Thanks

[livechemistryfunda]

I have a data for which I want to apply masks on images. There are 538 images but 152 masks. What to do? How to get masks for others?

[newcooldiscoveries]

Nice tip! A rarely mentioned but, I would suppose, a frequent and confusing problem. Thanks.

[rameshwarsingh]

Sreeni Sir, your videos are so great and helpful, ,Thank you sir

[jyothir]

Hello Sir, can u please do a video on image data analysis before training. What all we should consider before selecting an architecture, optimisers, loss etc to train on the data?

[sergiomorellmonzo]

I would like to make an appreciation about the masks in the remote sensing data which have some singularities. For semantic segmentation problems in satellite images (land use / land cover classification) we usually have a very large image that can be divided into small splits to create a set of smaller images (and with the same dimensions) and their respective masks. . However, ground truth data is usually obtained from field visits where some areas (eg. agricultural parcels) homogeneously distributed throughout the territory are labeled. This means that when creating the masks, not the entire image is labeled, since only some pixels are labeled. In this way we obtain masks with 3 classes of pixels: (1) class 1 (eg. Urban), (2) class 2 (eg. Forestry), (3) class 3 (eg. Agricultural) and (0) not labeled. How can we deal with this problem? Thank you very much for your help and your amazing explanations.

[gunawannurcahyo]

hello sir, this great video and documentation but i'm sorry, can i get the data standstone in this practice ?

[hammao]

Thanks for always sharing these wonderful tips, if you don't mind... what software do you normally use for creating your mask label. I'm working with geological materials and will like to generate label for a very large dataset.

[manikandanj]

Hi Srini, again thanks for the wonderful informative session.. if we want to hand annotate the ground truth from images for semantic segmentation, is there any tool?. can you make a video on it.

[patilvinod]

Sir, please make video on Metrics for evaluating 3D medical image segmentation like
Dice Coefficient
Jaccard Coefficient
Area under ROC Curve (one system state)
Cohen Kappa

Volumetric Similarity Coefficient

Hausdorff Distance (optionally in voxel or millimeter)


Average Hausdorff Distance (optionally in voxel or millimeter)
Balanced Average Hausdorff Distance
Mahanabolis Distance

Global Consistency Error
Coefficient of Variation
Probabilistic Distance
and also Classic measures

[sithibanu]

Thank u sir, it is very helpful for my research sir

[XX-vujo]

Why these codes only work with the dataset you have?

I am trying to do segmentation from colonoscopy images from the kvasir dataset. It does not seem to work and the dataset shape is only giving (808, ) can you help? Of do you have a video about it already? I am getting left behind with your videos. I am a slow learner.

[neelotpaltripathi]

sir how did you create these multicolour masks for these images

[aishwaryashrestha]

how can we use pretext task ? for any downstream task

[pranabsarma]

Hello Sir, Great video series. Its really helpful. Thank you for your time. Sir, I have a request. Can you please create a Discord server or Telegram Group where we all can connect and discuss about Image processing, Python, Machine Learning and Deep Learning.

[richardfinney]

Can you advise how to create masks from a dataset of images?