Speed Up Your Pandas Dataframes

In this video Rob Mulla teaches how to make your pandas dataframes more efficient by casting dtypes correctly. This will make your code faster, use less memory and smaller when saving to disk or a database.

Timeline:
00:00 Intro
00:47 Imports and Data Creation
02:32 Dataframe Memory Use
03:20 Baseline Speed Test
04:15 Casting Categorical
05:45 Downcasting Ints
07:07 Downcasting floats
08:15 Casting Bool Types
09:15 Benchmark Comparison
11:08 Outro

#python #code #datascience #pandas

[anoopbhagat]

That's a brilliant way to save memory & computational cost. Thanks Rob ! it was very useful.👍

[shreyaskulkarni]

I really predict that this guys channel is gonna grow a lot.The content is pure without any bs and straight to point with actually new info

[gilzeevi]

Listen Rob, i came across your channel pretty randomly and your content is pure gold! straight to the point, and professionally presented! Thanks a lot! keep rocking

[FilippoGronchi]

I love all the videos with these tricks that are critical in the daily developer activities! thanks so much

[Cmax]

Wow, this is one of the things that we rarely encounter in courses and yet the impact matters so much for overall efficiency. Thank you for making these types of videos. Wish this channel the best!

[shrekinahell]

this is incredibly useful information and explained nicely! subbed

[edmundgoldsberry]

Dude, where were you when I was starting out... you would have saved me hours of struggle. Great content. Please keep it coming.

[loctranp]

It's just wow! Beginner like me really appreaciate your video. Keep it up my man.

[clibonthegrind]

Very nice videos mate! Thanks for sharing your knowledge with us

[rakeshkumarkuwar]

Thanks Rob, for sharing such a great concept.

[TheRecordedLife]

Fantastic Video, definitely using this in my daily work.

[mohamedhhamed]

Thx Rob, really enjoyed this episode 👍🏼

[SimpleExcelVBA]

I'm surprised how much info I learnt from this video, really good work!

[gatorpika]

Great explanation. Just found out about that a while ago and wish I would have seen this video first instead of doing a bunch of googling. Was trying to get a 36M record dataset with categoricals and positional data to fit in memory on an average laptop for a mapping application. Recasting the datatypes made it all work out.

[deepsajwan]

Thanks Rob! for making this useful video

[beethovennine]

Nice vid! Glad I ran into your channel...subscribed!

[maxwellarnold]

I deal with stock data for my job a lot- dealing with data frames that have daily data for 3000 companies across 20+ years means dealing with 16+ million rows. These tips are incredibly helpful for saving memory- which for my role is often the limiting factor of pandas and my computer. Too much memory load can slow down groupby calls, your computer as a whole and all code, and even worse crash your computer which has happened to me.

[rajeshkakawat]

Thanks it usefull, will apply in my project

[bongkem]

awesome man, gonna save me $$$$ by not upgrading cpu but downsizing code !!!

[skarevaara]

Very nice video, thanks for the tips! If you do an update you could talk about unsigned integers also, like "uint8" for the Age data?