Stop Wasting Time on Simple Excel Tasks, Use Python

This video was sponsored by Brilliant

If you are new, welcome! I am John, a self taught Python developer working in the web and data space. I specialize in data extraction and JSON web API's both server and client. If you like programming and web content as much as I do, you can subscribe for weekly content.

:: Links ::

:: Disclaimer ::
Some/all of the links above are affiliate links. By clicking on these links I receive a small commission should you chose to purchase any services or items.

[mathewrtaylor]

Great tips! A suggestion as an addition, I would add to help with the potential memory issues of all the data frames is to import gc, and in your processing loop add:
del [[df]]
gc.collect
This will delete and remove from memory the df dataframe since you've already appended it to data in each iteration. This has saved me several times with memory issues. Hope this helps!

[AliceShisori]

thank you for yet another highly-applicable video John. just out of curiosity since you made this video and I know you are more well-versed in data extraction, do you plan to make anything else about excel and data manipulation :D ?

[oOBenictusOo]

I usually find your videos top-notch, but this case can also be permanently solved with Power Query in a maximum setup time of 5 minutes... looking forward to your next video, cheers.

[eziola]

Keep doing data cleaning examples in Pandas! Could be a lot of views for videos like these. Great video! Thank you

[tridibbiswas]

Thank you the video. These simple tutorials help people like to grasp some of the rudiments of the learning a programming language like python. BTW could you share the like to your dot files for nevoim. Would like to use of the stuff seen in your nevoim and figure out how to do it

[tmb]

I'm sure you know this by now, but the Polars read_csv method supports glob patterns, so the loop approach is unnecessary – you can simply pass _folder + "/*.csv"_ as the source parameter and the concatenation will be done automatically.

[adarshjamwal]

Great job john thanks for this content. I think your channel may be grow more. If you make and put another types of content like django, flask, javascript like that.

[Kiriletto]

Great vid. Can you tell me why, after you have appended the files in the loop (which I assume merges them together within the list) you then concatenate them into a new data store (ie the result)? Aren’t you combining them twice? Thanks

[markbennett]

Even more powerful once Python in Excel is released to mainstream

[acharafranklyn]

I noticed you dont like using ipynb content

[MichaelBrown-lwkz]

Any good Excel user would know not to "Copy and Paste" when preparing reports.

[christiancoronado]

Excel is a fast, simple solution for 90% of people out there. Why overcomplicating things.