Stopping valueless papers from getting into top journals | Spencer Greenberg (2023)

_Originally released March 2023._ Can you trust the things you read in published scientific research? Not really. About 40% of experiments in top social science journals don’t get the same result if the experiments are repeated.

Two key reasons are ‘p-hacking’ and ‘publication bias’. P-hacking is when researchers run a lot of slightly different statistical tests until they find a way to make findings appear statistically significant when they’re actually not — a problem first discussed over 50 years ago. And because journals are more likely to publish positive than negative results, you might be reading about the one time an experiment worked, while the 10 times was run and got a ‘null result’ never saw the light of day. The resulting phenomenon of publication bias is one we’ve understood for 60 years.

Today’s repeat guest, social scientist and entrepreneur Spencer Greenberg, has followed these issues closely for years. To do his bit to help get these numbers up, Spencer has launched an effort to repeat almost every social science experiment published in the journals _Nature_ and _Science_, and see if they find the same results.

Chapters:
• Rob’s intro (00:00:00)
• The interview begins (00:02:16)
• Social science reform (00:08:46)
• Importance hacking (00:18:23)
• How often papers replicate with different p-values (00:43:31)
• The Transparent Replications project (00:48:17)
• How do we predict high levels of success? (00:55:26)
• Soldier Altruists vs. Scout Altruists (01:08:18)
• The Clearer Thinking podcast (01:16:27)
• Creating habits more reliably (01:18:16)
• Behaviour change is incredibly hard (01:32:27)
• The FIRE Framework (01:46:21)
• How ideology eats itself (01:54:56)
• Valuism (02:08:31)
• “I dropped the whip” (02:35:06)
• Rob’s outro (02:36:40)

----

_The 80,000 Hours Podcast_ features unusually in-depth conversations about the world’s most pressing problems and what you can do to solve them.