Power Analysis, Clearly Explained!!!

If you're doing an experiment, a Power Analysis is a must. It ensures reproducibility by helping you avoid p-hacking and being fooled by false positives.

NOTE: This StatQuest assumes that you are already familiar with the concept of Statistical Power, Population Parameters vs Estimated Parameters and p-hacking. If not, check out the 'Quests:

For a complete index of all the StatQuest videos, check out:

If you'd like to support StatQuest, please consider...

Support StatQuest by buying The StatQuest Illustrated Guide to Machine Learning!!!

...or...

...a cool StatQuest t-shirt or sweatshirt:

...buying one or two of my songs (or go large and get a whole album!)

...or just donating to StatQuest!

Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:

0:00 Awesome song and introduction
0:42 Why we do a power analysis
2:40 Power analysis defined
3:14 Two factors that affect Power
4:06 How sample size affects Power
11:48 How to do a power analysis
15:10 Review of concepts

#statquest #statistics

[statquest]

NOTE: This StatQuest was brought to you, in part, by a generous donation from TRIPLE BAM!!! members: M. Scola, N. Thomson, X. Liu, J. Lombana, A. Doss, A. Takeh, J. Butt. Thank you!!!!

[JimtheEvo]

This video makes me feel so happy. I've had a paper in review for a little while, the editor and one reviewer noticed in one of my figures (in which I had shown the raw data and CIs) that a single datum point was not in line with the rest (n=6 not great but better than most molecular biology papers!). They asked me to add an extra replicate to the experiment. I refused on the grounds of p-hacking and showed them a power analysis (+93% power including the variation of the point) and showed the point is a Grubbs outlier. I'm still waiting for the editor to get back to me ;)

[raycyst-kv]

Man josh i am trying to finish your playlist . Before this i had tried khan academy, brandon foltz, few books and i failed horribly. P value was something i could never grasp . I think the issue was that everyone talked about what is p value but never explained HOW it is calculated. You really go in the depth of how and why. And this one video right here has blown my mind. It just doesnt explain power analysis but also how you estimate population mean and how p values work . It's 5 am right now and i am still studying .Youve really blown my mind!

[peterkilindberg]

Wow! I'm very impressed by your very clear explanations and calculations in this video. It was very easy to understand and follow. Big thanks from a PhD-student in Sweden!

[mathiasmadsen]

You've freaking done it again. I can't belive how simple this explanation was compared to the lecture I got in school. Thank you very much!

[tanishadorn]

Thank you! For my paper my lecturer said we don’t have to include a power analysis, so it wasn’t taught… so glad I watched this because I was struggling to justify my sample size and now it all makes so much more sense.

[uberdonkey]

I've done so much stats but akways had a difficulty understanding power analysis. This is very clear, and practical.

[sushanthraj]

I am so happy to see the growth rate of Josh Starmer's StatQuest. I still remember I was one of the few subscriber who joined your fan base when the subscribers where in thousands!. Great going happy learning

[vladfarias]

Your videos are absolutely amazing! It would be fantastic if you could create (if it doesn't already exist) a video covering the entire hypothesis testing process, including all the steps. That would involve determining the sample size and addressing the temptations encountered along the way until reaching the final result.

[SubDonkess]

Fantastic! I’m taking a statistical modeling class with machine learning and this particular topic just wasn’t sticking with me. As advertised, this was super clear and nailed home all the key points!

[khoiavo]

I learned more (and laughed more) in these 16 minutes than a whole semester at med school. Thank you!

[efrensuarez]

One of the best channels out there to learn statistics.

[MrDrache]

Josh in May 2020: "Imagine there is a virus"
2020: "Hold my beer"

[nikhilgoyal]

Josh, thanks so much for your videos - so clearly explained relative to other content I've seen on inference. I do have a few questions.

1. In an A/B testing scenario, we don't know the mean and the standard deviation of the distribution of the treatment group before hand. We do know the mean and the standard deviation of the prior distribution. If I want to estimate sample size required for different effect sizes, holding the chosen p-value threshold constant at 0.05 and power at 0.8, how would we do that given that we don't have s2 or standard deviation of the second group?

2. Also since we don't know the mean of the treatment group's distribution, how would we calculate the estimated difference in means to plug it into the formula for effect size?

3. Do you have a video on How to pick the right test?

[faye]

Thanks Josh, you are the best at storytelling when explaining statistics.

[zahrarezazadeh]

Hi Josh, thank you a lot for the awesome videos!
I have two basic questions:
1. The denominator for the pooled estimated SD in a general condition is the number of the distributions (and not always 2), right?
2. What is the deal with statistics power calculator? isn't there simple formulas we can use ourselves? and does the googling we do to choose one, involve the nature of our experiment, or we just randomly choose one?

[JS-wvvn]

Thanks I haven’t slept this good in weeks

[PrabhakarKrishnamurthyprof]

Thank you Professor. I will use it for my class. It is so well explained, I learnt how to explain complex concepts.

[taotaotan]

Simulation is really helpful to understand this.

[pandapanda]

i have a question. so knowing statistical power, significance level, effect size, and sample size are all related through power analysis, then wouldn't choosing a statistical power (or even significance level, as we know 0.05 threshold is just for teaching demonstration and in reality it can be any different amount) also be consider p-hacking by effectively getting a desire sample size in "remote" in order to have it prove (or disprove) hypothesis at the researcher's discretion? or is there an objective way to determine the statistical power (and significance level)? most of the materials i read often says "commonly use power = 0.8 or alpha = 0.05" or even heard "amount chosen at researcher's discretion" but without giving sufficient reason for picking those amount.