How to Detect and Fill Missing Values in Pandas (Python)

This video shows how to detect and fill missing values such as NaN, NA, None and the empty string in Pandas data frames. Detecting, counting and filling missing values or other odd values is a basic data exploration and cleaning step that is going to be necessary with all but the cleanest real world data sets.

If you find this video useful, like, share and subscribe to support the channel!

Code used in this Python Code Clip:

import numpy as np
import pandas as pd

mtcars["None_col"] = None

# Count the total number of missing values

missing_vals = ["NA", "", None, np.NaN]

# Fill null values (NaN and None) with a given value:

# Fill a list of missing values with a given value:

missing_vals = ["NA", "", None, np.NaN]

* Note: YouTube does not allow greater than or less than symbols in the text description, so the code above will not be exactly the same as the code shown in the video! I will use Unicode large ＜ and ＞ symbols in place of the standard sized ones. .

[KhalilYasser]

Amazing tutorial. Thank you very much.

[bennguyen]

How would you fill empty values (NaN or NA) in a list or dataframe with the last known good value?

[simonaneaga]

Once you have detected the missing values, would it somehow affect the regression model?

[sakinaaali]

how about if i replace the missing values with an average value? can anyone help me with the code?

[SoulSheIS]

Thank you for the video.

In practice, I believe tried this (with using the numpy library)using isnull() .sum(), missing string values aren't detected.

It counts the entire dataframe as having values.

It is only when writing to a csv file, I detected this error.

How do you detect missing values, when they are present, but undetected via the former steps?

[yunusemreylmaz]

Thanks for all the videos it helped me a lot. However I searched on google a long times but I could not find my problem. I am trying to fill missing values with others columns. I mean there are some missing values about cars body type but there are information about body type in another column.