Vectorization in Python : Data Science Code

Crazy speedups with vectorization in Python.

[jordanfogel]

this is absolutely fantastic and so clearly and well-explained, much appreciated man!

[ShaunakDe]

Thanks for putting this out. This is such an under-rated skill with so few resources out there

[kepstein]

You are a great teacher. Very intuitive and down-to-earth. Thank you.

[vinson]

This concept is very important, but I see a lot of people who use Python don't know the concept of vectorization mainly because they come from C/Java/any other compiled languages where vectorization is the job of the compiler

[agustindangelo]

Thanks ritvik, your videos are being pretty helpful for me! Great explanations.

[alsonyang]

Thanks for the amazing video. Concepts are well-explained.
However, there is one possibly misleading part, which is vectorization != parallelization on the core level. For vectorization, the parallelization is actually done within one core on the register level. To prove this, try running the NumPy code in the demo, you would see the one CPU that python is running on, will boost up to 100% usage, while all the other CPUs are idled.
Except for that, I think everything else is very clear and helpful. Thanks again!

[diegoa.]

Great video! Everything was so clear and well explained

[smudgepost]

Such an excellent video thank you for giving such a clear example. After struggling with a very slow and laggy loop earlier today I was trying to learn about multi-threading and also multiprocessing to see if I could speed up the calculation noting that CUDA is not supported on my current GPU. Then I found one of your web posts and was reading the code and testing it out without fully understanding a vectorization. To get more information I searched on YouTube later in the day and what did I find? I found the same person with the detailed video! Thank you so very much. I'm really getting my head around how natural language processing handles vast arrays of data so quickly and I believe my future is vectorization..

[prakashsuryakumar]

Thank you very much for the content. I really appreciate the amount of work and time that goes into making such videos.

[jimmylyu]

it makes so much sense now! Thanks for the video

[jessicatran]

love it! so easy to follow! thanks for your videos!

[Veebop]

Summed up:
Its still iterative, but using parallel conputation + not being strict in the Order of computation.

Will numpy always use all 12 cores If i have a 12 cores cpu?

[haneulkim]

Amazing content! I love how you explain reasons why vectorization is faster. So parallelization does not equal vectorization however it is one characteristic of vectorization, right?

[jakubkahoun]

Thanks you VERY VERY much. I m self tought, and this speed up my home project about 2000x :D

[mattkriese]

So cool. Even as a newbie I was able to follow. Cheers 🇨🇦

[messi]

Quick follow up question: are there any other libraries in python apart from numpy that leverage vectorization?

[deanmait]

Great tutorial. Not sure the matrix products are as close though - i ran a few examples of the same code and the np.mean(abs(C2-C1)) difference was close to 2

[arjungoud]

you are awsome man. Can you please take some time, and explain ReLU vs Sigmoid.

[spearchew]

good videos but i felt the reference to multiple cores in the first 2 minutes was confusing. if i understood correctly, the major speed up is that you can do operations on a massive list in one go rather than element by element... and this is true even if you have one core... (?)

[Johnspeed]

This is might be a stupid question. When he ran the 'for' loop and gave him about 7 secs to run, the actual time it took was about 2-3 seconds to run. Is this '7 secs' different units than normal seconds? What am I missing?