Complete Python Pandas Data Science Tutorial! (Reading CSV/Excel files, Sorting, Filtering, Groupby)

Practice your Python Pandas data science skills with problems on StrataScratch!

Let me know if you have any questions!

In this video we walk through many of the fundamental concepts to use the Python Pandas Data Science Library. We start off by installing pandas and loading in an example csv. We then look at different ways to read the data. Read a column, rows, specific cell, etc. Also ways to read data based on conditioning. We then move into some more advanced ways to sort & filter data. We look at making conditional changes to our data. We also start doing aggregate stats using the groupby function. We finished the video talking about how you would work with a very large dataset (many gigabytes)

I realized as I upload this video there are some additional things I want to talk about in a later video. The first thing that comes to mind immediately is using the apply() function on a dataframe to alter the data using a custom or lambda function. If you have questions on this or anything else before I get around to making a part 2, feel free to write me a note in the comments.

If you enjoyed this video, be sure to throw it a like and make sure to subscribe to not miss any future videos!

Thanks for watching friends! Happy coding! :)

Join the Python Army to get access to perks!

---------------------------------------------
Follow me on social media!

---------------------------------------------

---------------------------------------------
Video Outline!
0:00 - Why Pandas?
1:46 - Installing Pandas
2:03 - Getting the data used in this video
3:50 - Loading the data into Pandas (CSVs, Excel, TXTs, etc.)
8:49 - Reading Data (Getting Rows, Columns, Cells, Headers, etc.)
13:10 - Iterate through each Row
14:11 - Getting rows based on a specific condition
15:47 - High Level description of your data (min, max, mean, std dev, etc.)
16:24 - Sorting Values (Alphabetically, Numerically)
18:19 - Making Changes to the DataFrame
18:56 - Adding a column
21:22 - Deleting a column
22:14 - Summing Multiple Columns to Create new Column.
24:14 - Rearranging columns
28:06 - Saving our Data (CSV, Excel, TXT, etc.)
31:47 - Filtering Data (based on multiple conditions)
35:40 - Reset Index
37:41 - Regex Filtering (filter based on textual patterns)
43:08 - Conditional Changes
47:57 - Aggregate Statistics using Groupby (Sum, Mean, Counting)
54:53 - Working with large amounts of data (setting chunksize)

-------------------------

*I use affiliate links on the products that I recommend. I may earn a purchase commission or a referral bonus from the usage of these links.

[KeithGalli]

Hey ya'll! I created a second channel with more Python content (including additional Pandas tips & tricks).

Please consider subscribing 😊

[jcspaziano]

I know this is 5 years old but I learned more about using Pandas from this one video than all the other videos ive watched on the topic combined! Just awesome! Thank you!

[RisingLoaf]

This 1 hour video did more for me than entire semester of my Data Analysis course... Amazing

[dicspringdkz]

Keith
You are more than a teacher. Your level of simplicity in explaining Python in details is out of the moon. Keep up the good work. Your video is always my “go to” any time.
Again, thanks a lot for using your skills as a blessing to people around the world.

[_Nelyen]

This video was super helpful, thank you Keith!

In case anyone gets to the end of this video, around 48:00, Keith talks about the groupby operator and starts to go over the section "Aggregate Statistic using Groupby (Sum, Mean Counting)". You might run into errors due something that changed after Pandas version 2.0.0.

Instead of writing: df.groupby(["Type 1"]).mean()
Try writing: df.groupby(["Type 1"]).mean(numeric_only=True)

After version 2.0.0 the numeric_only value was changed to False versus True as it's default, causing errors such as "can not convert strings". Hope this is helpful, have a good one!

[LureUnitFtw]

One of the best tutorial that I've ever seen in YouTube! Thumbs UP!

[Orionk]

Mannnn your one of the best Python go-tos PERIOD. Straight to the point and easy to understand. thanks for teaching us all!

[piotr]

Hi Keith - not sure you will read this but wanted to sincerely thank you for this tutorial. 3 years ago this was the first python video I ever watched after graduating from unrelated subject. Today I'm typing this from a business class lounge at JFK, on my way to London where I just got a job as a quant developer at a hedge fund, building pricing models and infra for trading. Worked hard for this but if not for your videos I could be at a very different place. Thank you from the bottom of my heart, your work means a lot to many people. Cheers!

[nikithroumpari]

A strugling biologist here thanks you! We are mostly dealing with big data and it can get a little overwhelming, but you made it a lot easier!

[bentrash]

Awesome tutorial! One advice I'd have for any python developers is to get in practice of working within virtual environments. Really helps to avoid conflicts when you're working on a project which may require some older versions of a library but your other projects may require latest ones, stuff like that.

[klauscheang]

Excellent!! I like the way you organize the videos on different topics and functions of working with data. Please make more videos on how to work data science in Python. E.g. Statistical analysis (descriptive statistics, t-test, linear regression) or data processing tutorial (like what we do in SQL).

[faizalimuhammadzoda]

There is something to the way Keith teaches that keeps me coming back.
Besides being a good teacher and utilizing techniques which help people grasp the material quickly and remember for long time, he sends forth a wave of positivism. He is such a positive, energetic person.

Thanks for sharing your knowledge. May it grow and enable you to bless more people with it.

[hughjazz]

I have bought multiple Udemy courses on pandas and this one blows them all out of the water, and it’s free! I’m deff subbing!

[KeithGalli]

Video Outline!
0:45 - Why Pandas?
1:46 - Installing Pandas
2:03 - Getting the data used in this video
3:50 - Loading the data into Pandas (CSVs, Excel, TXTs, etc.)
8:49 - Reading Data (Getting Rows, Columns, Cells, Headers, etc.)
13:10 - Iterate through each Row
14:11 - Getting rows based on a specific condition
15:47 - High Level description of your data (min, max, mean, std dev, etc.)
16:24 - Sorting Values (Alphabetically, Numerically)
18:19 - Making Changes to the DataFrame
18:56 - Adding a column
21:22 - Deleting a column
22:14 - Summing Multiple Columns to Create new Column.
24:14 - Rearranging columns
28:06 - Saving our Data (CSV, Excel, TXT, etc.)
31:47 - Filtering Data (based on multiple conditions)
35:40 - Reset Index
37:41 - Regex Filtering (filter based on textual patterns)
43:08 - Conditional Changes
47:57 - Aggregate Statistics using Groupby (Sum, Mean, Counting)
54:53 - Working with large amounts of data (setting chunksize)

Thanks for watching friends! :)

Let me know if you have any questions

[nimaonta]

Dude you deserved all the subs for this video alone. You explained everything so good. keep it up :)

[DennisGorshteyn]

You break down all the details in a way that I can't believe this is for free. Very high quality stuff. I was up and running with this library in short order

[adedokunagunbiade]

I watched the entire video in 30 minutes and learned more than I did with hours of video content. Amazing work.

[AndrewMann]

Between jobs for the first time in decades I wanted to learn data science using software other than just Excel and Access. Your video was well explained and frankly better than anything else I have seen so far involving Python and Pandas. Thank you for a job well done.

[nikluz]

this is an excellent tutorial, especially the filtering/conditional changes section. I have always loved how google sheets has built in queries, and I wanted to be able to do a lot of the same things using pandas. This essentially gave me all of the power I needed! thanks!

[kanstantsinhupalau]

Saved my day! I started learning Pandas, but when I missed several months during circumstances and this video about basics helped me quick comeback. Thank you!