Correcting Skewed Data with Scipy and Numpy

Показать описание

Skewed data can adversely affect your analysis and machine learning models. In this video, I demonstrate five methods for cleaning skewed data using the NumPy and SciPy modules. The methods include taking the square root, cube root, fourth root, log, and Yeo-Johnson transform. I also showcase the effectiveness of each method by summarizing the skewness of the data after each transformation with a bar plot.

Рекомендации по теме

Комментарии

First time I have seen your videos. This is genuinely a very good video. Very well explained and clear. I am subscribing.

The music wasn’t off putting either!

marcom

the on-screen text is a great addition, Dr. P!

officialscience

Thank you for the video, subscribed! Youtube needs more quality content like this.

metinunlu_

Amazing video
I like it's structure: motivation, overview with examples, practical advices
Thanks!

mmucyex

So what about if we were to standardize using z-scoring? It seems like that would get largely the same impact, wouldn't it?

undertaker

This is interesting. If one corrects the original skewed data, via doing these kinds of transformations, in the context of linear regression or multiple linear regression, will that not change the interpretation of the original data. Curious to know.

pabloagogo

Skewing doesn’t necessarily matter if you’re using XGBoost, correct? For classification or regression, that is

AyahuascaDataScientist

Amazing video! I was creating a function for measuring the same you forgot to name log1p Wich is log of (x+1) really useful for right skewed data with values less than 1

nicolaslpf

Thank you! That was helpful!
So we basically can make the root of any power? Is there a drawbag for exploiting it, like keep increasing the n value for feature to the power of 1/n?

dannybee

SUBSCRIBED! What should one do before? Or, what's the correct order? - treating outliers, impute missing values, correct symmetry? Thanks Dr. P!

thoniasenna

What if my data contains a lot of useful '0' values?

pewkaboo

Bro you explain a concept, but go you need the music!! It’s distracting

mouhsineelqesry

Correcting Skewed Data with Scipy and Numpy

Correcting Skewed Data with Scipy and Numpy

Handling skewness

Correcting skewed data with scipy and numpy

Correcting skewed data with scipy and numpy

Gaussian data distribution, skewness, and Kurtosis with Python | Log and Boxcox power transformation

Log Transformation for Outliers | Convert Skewed data to Normal Distribution

Skewness and Kurtosis in Statistics | What is Skewness? | Handle Skewness | Satyajit Pattnaik

Calculating Skewness by SciPy LECTURE75

PYTHON : How to correctly use scipy's skew and kurtosis functions?

(IS04) Skewed-Data Analysis

7 Statistics tutorial | Skew in python statistics | Kurtosis in python | Data science

Feature Engineering-How to Transform Data to Better Fit The Gaussian Distribution-Data Science

SPSS Tutorial: Transforming asymmetrical/skewed data

Skewness And Kurtosis And Moments | What Is Skewness And Kurtosis? | Statistics | Simplilearn

When should we use which Transformation to correct Skewness ?

Standardization vs Normalization Clearly Explained!

Skewed Data Analysis Part 1

Skew and Kurtosis - Stats with Python

Finding an outlier in a dataset using Python

V21: Cross-Sectional Skewness and Kurtosis in EU BlueChips (using Python)

Outlier Treatment through Square Root Transformation | Convert Skewed data to Normal Distribution

Positively Skewed Distribution & techniques to reduce skewness

Python - Histograms and Skew Made Easy - Part 1

Statistics in Python: Transform a non-normal distribution to a Gaussian distribution.