Python Threading Beginners Tutorial - Threading Example #1 Image Downloading

Python Threading Beginners Tutorial - Threading Example #1 Image Downloading

This video demonstrates the benefits of using Python threading to dowload images.
This video is a continuation of the threading playlist, so it's highly suggested you
look over the previous material before you watch this video:

SEE UPDATE TO THIS VIDEO HERE:

This update shows how to fix the "hanging" problem from this video

Code can be found here:

Please comment on how multi-threading from this example is working for you guys on large datasets

Thank you.

Twitter @Moondra2017

[PyMoondra]

The hanging error was fixed in a follow-up video:

[rgonnering]

Moondra, These threading videos are excellent. Thank you.

[davidbristoll]

Another hugely helpful video, thank you. With regard to it hanging predictably, particularly at 198, I have a theory:

You get errors for images that can no longer be downloaded, so they are skipped pretty quickly. However, if an image is particularly large in size or the website is particularly slow, it's still a successful request, it just takes a lot longer.

I think that if you could identify the culprit images, this would run like lightening.

Hope that's it. Anyway, huge thanks for these very helpful videos. I'll be implementing this approach in my work!

[raghavanp]

Hi, I think It could due to number of open file descriptors limit in your machine. Also, as we increase the number of threads, they don't release the http connection . ie, your connection pool gets filled and no more open channels available to create new connection. With = 1, we can probably see how the connection pool limit decreases gradually. Thanks for the nice tutorial.

[devashishgupta]

why have you not used lock for the image_count variable, it is accessed by multiple threads simultaneously?

[sajjadhossan]

I have also accessed your itertools playlist. I want to learn some more python build-in modules from you.

[sakshamrajbhandari]

why does it say "looks like the image format isn't supported"? Please help!!

[Denverse]

Thank you sooo much... Your videos are so helpful..

[inaramara]

please Mr...why we use q.join() you did'nt mentioned i guess in the queue video ....and thank you ...you have the most powerfull channel in python field

[rgonnering]

Moondra, Is it possible for you to post the list of image urls? Thanks.

[georgesmith]

I use the urlretrieve and it doesn't seem to get stucked when there is an error.

[xtremehackerzpro]

Why do you don't use Sublime or Atom...